Clinical Risk-Weighted Evaluation of ICU Mortality Prediction Models on MIMIC-IV

<title>Abstract</title> <p>In ICU mortality prediction, a False Negative (FN) error—a missed death—carries a graver clinical consequence than a False Positive (FP) false alarm. However, models are routinely evaluated using the F1 score, which penalises FN and FP equally. We propose the Clinical Risk-Weighted Score (CRWS), a twoparameter generalisation of the F-β score that decouples FN and FP penalty weights for clinical interpretability. CRWS reduces exactly to F-β when wFP=1 and to F1 when wFN=wFP=1. We evaluated four classifiers on in-hospital mortality prediction using MIMIC-IV v3.1 (65,366 patients, 10.84% mortality). To isolate metric behaviour, models were trained on static tabular features. Results show model rankings shift significantly under CRWS: Random Forest achieves the highest accuracy (87.23%) but misses the most deaths (FN=1,220), ranking last under CRWS (0.147). XGBoost ranks first under both metrics, but CRWS (0.596) reveals a larger clinical advantage than F1 (0.298). We demonstrate the tunable effect of wFP and confirm pairwise model differences are statistically significant (McNemar’s test, p &lt; 0.001). Sensitivity analysis, threshold optimisation, bootstrap confidence intervals (n=500), and a SMOTE ablation study confirm the robustness of these findings under asymmetric error costs.</p>