Where Did the Resistance Go?
How Empirical Claims Survive Evidence That Should Weaken Them
Jake Lee · Red Analytics, Inc. · August 2026
Where Did the Resistance Go? introduces resistance handling: ways evidentiary resistance can be treated so that it fails to carry forward into the claim. The framework is intended to help readers recognize when evidence should reduce confidence before a claim is relied upon, reused, or presented.
Abstract
Sometimes the evidence in a paper pushes against its main claim. The failed test is visible. The disappointing validation is reported. The unfavorable comparison is acknowledged. Yet by the time the reader reaches the abstract or conclusion, the claim has barely changed.
How did that happen?
Resistance handling refers to ways evidentiary resistance can be treated so that it fails to carry forward into the claim. The framework identifies recurring patterns including rhetorical resolution, evaluation refactoring, missing expected baselines, selective metric invalidation, and semantic inflation.
The contribution is classificatory rather than corrective. It provides language for recognizing when restraint is warranted before a claim is relied upon or reused.
Five Patterns of Resistance Handling
Rhetorical Resolution
Contrastive language such as “although,” “however,” or “despite” does the interpretive work needed for a claim to survive largely unchanged, without additional evidence that earns that move.
Evaluation Refactoring
The way evidence is evaluated changes after the original or expected evaluation resists the claim—for example through a new metric, aggregation, transformation, or subset of the evidence.
Missing Expected Baseline
A claimed advantage is evaluated without the standard or naturally expected comparison needed to interpret that advantage.
Selective Metric Invalidation
A metric is challenged where it produces resistance, while favorable uses of the same metric remain largely untouched.
Semantic Inflation
Familiar methodological language associated with validation, prediction, replication, or generalization gives evidence more credibility than the underlying procedure earns.
1. Purpose and Scope
The standard addresses one narrow problem: the evidence pushes against a claim, but the claim does not weaken.
This standard arose from recurring patterns observed in published empirical and methodological literature. In those cases, evidence initially resisted a claim, yet the resistance disappeared from the interpretation without a corresponding weakening of the claim. The patterns described here are inductive rather than exhaustive. They are not intended to estimate how often these practices occur.
This standard is not a critique of any method, author, or domain. It is not a proposal for best practice, formal pre-registration, or ideal science. Its purpose is narrower: to provide standards for downgrading belief in empirical claims when resistance from the data disappears without a corresponding weakening of the claim. The standards operate at the level of interpretation, framing, and evidentiary handling, not estimation.
These standards are intended for readers responsible for deciding whether empirical claims are safe to rely on, reuse, or present, and incidentally for those tasked with producing the original analysis. A classification under this standard describes how evidence, evaluative criteria, and claims relate to one another. It does not establish intent, misconduct, or falsity.
Related problems include HARKing (Kerr, 1998), spin (Boutron et al., 2010), and researcher degrees of freedom (Simmons, Nelson, & Simonsohn, 2011). Resistance handling is narrower. It asks what happens after evidence pushes against a claim and whether that resistance carries forward into the interpretation.
2. The Core Question
The problem begins when evidence does not behave as a claim requires.
Empirical claims ask the evidence to clear particular hurdles before belief is warranted. Those hurdles may come from standards stated in the analysis, established evaluative standards for that kind of claim, or something the claim itself requires. When evidence fails to clear a relevant hurdle, it resists the claim.
Resistance refers to points where evidence does not behave as required for belief: tests that fail, validations that disappoint, claimed advantages that do not survive relevant comparisons, or instabilities that appear under reasonable variation. Resistance is not error or misconduct. It is ordinary friction between data and claims.
Resistance can be treated in many ways. A failed test may be explained, reframed, reinterpreted, or otherwise treated in a way that changes its apparent importance. When that treatment prevents the resistance from carrying forward into the claim, it becomes resistance handling.
If the evidence resists, the claim should show it.
3. The Symmetry Check
The next question is whether the treatment applies symmetrically.
When evidence resists a claim, explanations are often offered. Some explanations change how the evidence should be understood generally. Others apply only to the unfavorable result in front of us.
The symmetry check asks whether the same concern would apply equally to favorable and unfavorable evidence. If it would also require reconsidering favorable results that depend on the same criterion, it is doing real evidentiary work.
If the same concern matters only when the result is unfavorable, the treatment is asymmetric.
4. Patterns of Resistance Handling
The patterns below describe recurring ways a claim can remain stronger than the evidence or expected comparison warrants.
4.1 Rhetorical Resolution
Rhetorical resolution occurs when contrastive language does more work than the evidence in front of it.
Common forms include:
- “Although the results suggest X, the interpretation is not straightforward…”
- “Despite these limitations, the results support…”
- “Initial analyses raise concerns, but subsequent considerations suggest…”
The problem is not the language itself. Contrastive language often improves understanding. The concern arises when a “but,” “however,” “despite,” or similar construction supplies the interpretive move needed for the claim to survive largely unchanged, without additional evidence that earns that move.
The question is whether the contrast explains the evidence or rescues the claim.
4.2 Evaluation Refactoring
Evaluation refactoring occurs when the way evidence is evaluated changes after the original or expected evaluation resists the claim.
The change may involve a new metric, a different aggregation, a transformation, or renewed attention to a subset of the evidence. Any of these may be legitimate. The concern arises when the original or expected evaluation resists the claim, but the claim survives by being handed off to a refactored evaluation.
The refactored evaluation should have to earn the claim it is now carrying.
4.3 Missing Expected Baseline
A claimed advantage is evaluated without the standard or naturally expected comparison needed to interpret that advantage. The reported comparisons may be valid on their own, but they cannot establish superiority relative to a baseline that was never included.
The relevant baseline is not whatever comparison would make the claim hardest to support. It is the comparison that an informed reader would ordinarily expect given the claim, the established standard, or the practical alternative under consideration.
When the expected comparison is absent, the reader cannot see whether the evidence would have resisted the broader claim.
4.4 Selective Metric Invalidation
Selective metric invalidation occurs when a metric is challenged where it produces resistance, while favorable uses of the same metric remain largely untouched.
Metrics can have real limitations. The concern arises when those limitations become consequential only for the unfavorable result. If the objection is genuine, it should change how the metric is treated wherever it is used to support the same kind of inference. An alternative metric may be useful, but it does not automatically inherit the evidentiary role of the metric that was just invalidated.
A genuine objection to a metric should travel with the metric, not only with the unfavorable result.
4.5 Semantic Inflation
Semantic inflation occurs when familiar methodological language gives evidence more credibility than the underlying procedure earns.
Terms associated with validation, prediction, replication, or generalization often carry an understood evidentiary meaning. The concern arises when the procedure satisfies a weaker meaning of the term while the claim benefits from the stronger meaning readers would ordinarily infer.
The label does not upgrade the evidence.
5. Scope of Application
A resistance-handling pattern changes how much weight a claim should carry, not whether the claim is automatically false.
Downgrading belief is an action of degree. The appropriate downgrade depends on how much the claim would have to weaken if the resistance were carried forward. A minor qualification may change little. A result that would materially narrow or reverse the conclusion should reduce reliance much more.
Withholding belief is not the same as falsifying a claim. Exploratory work may remain useful even when the evidence is not strong enough to support adoption, reuse, or a broader conclusion.
Sometimes the correct response is simply not to rely on the claim.
6. Guardrails for Use
These standards should constrain skepticism as well as support it.
- Strong evidence may legitimately exhibit little resistance. The absence of resistance is not itself evidence that resistance was hidden or handled.
- These standards justify downgrading belief, not upgrading it. Passing the standard only means that this particular reason for withholding belief was not observed. It does not establish that the claim is true.
- The same standard applies when evidence resists the evaluator’s own prior belief. Explanations used to preserve a skeptical conclusion should survive the same symmetry check.
7. Closing
These patterns do not determine whether an empirical claim is true. They identify situations where resistance from the evidence has not been carried forward into the claim, and where reliance should therefore be reduced.
Claims that exhibit these patterns may still be explored, debated, or refined. Additional evidence may eventually support them. The standard only identifies when the evidence currently offered should carry less weight.
Resistance is ordinary. Handling it without carrying it forward can make a claim unsafe to rely on.
References
Boutron, I., Dutton, S., Ravaud, P., & Altman, D. G. (2010). Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA, 303(20), 2058–2064. https://doi.org/10.1001/jama.2010.651
Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
Citation and Full Paper
Lee, Jake. (2026). Where Did the Resistance Go? How Empirical Claims Survive Evidence That Should Weaken Them. Red Analytics, Inc.
DOI: 10.5281/zenodo.22048272