Abstract
<p>Causal machine learning methods for experiments have proliferated, but adoption in political science is scarce. Are political scientists "leaving money on the table" by neglecting to apply these methods? To answer this question, we use over two million Monte-Carlo simulations to compare causal forest (CF) to standard regression techniques commonly used to estimate average and conditional average treatment effects. We find that CF outperforms standard methods in terms of statistical power, coverage, precision, bias, and the accurate detection and consequent estimation of heterogeneous treatment effects. However, CF underperforms in edge cases where measured covariates are entirely uninformative and sample sizes are very small. Moreover, we find that standard approaches have large false-positive rates for conditional treatment effects: 9–24 times larger than rates for CF. Our results suggest that researchers who choose to use CF stand to benefit greatly across statistical power, coverage, precision, bias, and heterogeneity detection.</p>