Abstract
<p>The traditional approach of identifying response time (RT) outliers calculates exclusion thresholds across the entire sample of RTs, thereby neglecting any non-stationarity in RT such as practice related RT changes. Consequently, the proportion of a trial being excluded as outlier can be confounded with its position within the experimental session, e.g., as slow RTs tend to come more from the beginning of the experiment. Here, we demonstrate such undesirable behavior for a traditional exclusion approach based on a constant z-score (e.g., mean RT ± 2 × SD), for which outliers are more likely to come more from the beginning of the experiment the more trials were available. Critically, this constant method also introduced bias into the analysis of mean effects, and more specifically for effect estimates at the beginning of the experiment. Such bias could be reduced by an exclusion method based on moving z-scores, which identifies outliers according to z-scores obtained by a sliding window, therefore accounting for global changes in RT. Identifying outliers according to a moving threshold thus more appropriately accounts for the characteristics of evolving RTs, moving beyond the stationarity assumption inherent in the traditional approaches. R code for implementing this procedure is provided.</p>