Abstract
<title>Abstract</title> <p>Acute kidney injury (AKI) affects 10-50% of intensive care unit (ICU) patients and carries substantial morbidity and mortality. While machine-learning models for AKI prediction have proliferated, none provides distribution-free coverage guarantees on prediction uncertainty, a critical gap for clinical deployment. We present a conformal prediction framework for AKI in the ICU using 38,287 real ICU patients from a publicly available multi-center dataset. Models were developed on one hospital (n = 19,875) and externally validated on a second independent hospital (n = 18,412). We benchmarked six classifiers: logistic regression, random forest, gradient boosting, XGBoost, LightGBM, and a multi-layer perceptron (MLP), and applied split conformal, class-conditional conformal, and Mondrian conformal prediction. XGBoost achieved the best external AUC of 0.935 (95% CI: 0.930-0.940), outperforming the MLP (0.865). Marginal conformal prediction delivered 92.1% external coverage but under-covered AKI-positive patients (31.3%). We prove that class-conditional conformal prediction is exactly valid under a change in disease prevalence (label shift), so that any residual under-coverage on a new site isolates within-class distribution shift rather than the prevalence difference. Empirically, class-conditional conformal prediction raised AKI coverage to 87.2% (95% CI: 86.6%-89.8%); by our result the gap below 90% quantifies within-class shift between the two hospitals. Mondrian conformal ensured group-conditional coverage >= 89.6% across gender and age subgroups. SHAP analysis identified creatinine measurement frequency and BUN as top predictors. A conformal risk stratification partitioned the external cohort into Low Risk (78.1%, AKI rate 1.5%) and Elevated Risk (21.9%, 35.5%). Five-fold cross-validation confirmed stability (AUC = 0.926 +/- 0.009). To our knowledge, this is the first application of conformal prediction to AKI in the ICU with external validation.</p>