Abstract
<p>Simulation studies to date have examined the impact of Questionable Research Practices on the false positive rate of common statistical inference tests (e.g., Stefan &amp; Schonbrödt, 2023), but not the impact of unquestionably undesirable research practices, such as fabrication and falsification. While any result can be obtained through sufficient fabrication or falsification, we have little insight into how much of either is needed to generate statistically significant results under a null hypothesis. This article therefore simulates data generated under the true null hypothesis and alters, deletes, or invents one data point at a time, targeting the data points that most oppose the desired result, and re-running the test after every change until it is significant in the intended direction. Six tampering strategies (dropping cases, switching condition labels, re-pairing values, duplicating, fabricating, and replacing cases) are applied to three common tests: the independent *t*-test, Pearson *r* correlation, and the Chi-square test for a 2x2 contingency table. Results demonstrate that, in small samples, only a small absolute number of tampers are typically needed to obtain significant results. In large samples, although the absolute number of tampers needed grows, the proportion of the sample needing to be tampered with to obtain significance falls rapidly. The strategies that alter no recorded value, and so are plausibly hardest to detect, are among the most effective. Critically, unjustifiably deleting inconvenient data points, which researchers rate as far more defensible than invention, is no less dangerous than outright fabrication: both manufacture false positives from null data reliably, and at comparable and very small cost (e.g., median tampers of 5-10% at moderate sample sizes across tests). Fabricating and falsifying results therefore requires a worryingly small amount of actual data tampering.</p>