Abstract
<title>Abstract</title> <p> Background Programmatic assessment needs validity evidence to justify the design of multidomain formative systems. Yet we still know little about how cognitive, procedural, and communication assessments each predict examination performance within a single integrated course. This study examined the predictive validity of these domains in an undergraduate diagnostic medicine course that delivered all assessments through one online platform. Methods This retrospective cohort study included 382 undergraduate medical students (220 males, 162 females) in a diagnostic medicine course at Guangdong Medical University during the 2024–2025 academic year. One online learning platform delivered and recorded all formative assessments. The assessments covered three competency domains: cognitive (theory assignment and PBL), procedural (PE video assignment and PE skills assessment), and communication (SP interview). We used Spearman rank-order correlations, simultaneous multiple regression, and four-step hierarchical regression. Sex and cumulative GPA entered the models first as covariates. Results All five formative components correlated with written examination performance ( <italic>ρ</italic> = 0.19 to 0.44, all <italic>p</italic> < .001). After adjustment for covariates, PBL ( <italic>β</italic> = 0.20) and PE video assignment ( <italic>β</italic> = 0.18) were the strongest independent formative predictors (both <italic>p</italic> < .001). Theory assignment scores did not predict performance independently. This was consistent with a ceiling effect from the post-class MCQ format (mean = 89.04 ± 8.53, skewness = − 2.34). Each domain block added significant incremental validity: cognitive (Δ <italic>R</italic> ² = 0.049, <italic>p</italic> < .001), procedural (Δ <italic>R</italic> ² = 0.038, <italic>p</italic> < .001), and communication (Δ <italic>R</italic> ² = 0.006, <italic>p</italic> = .033). Together they explained 50.5% of examination variance beyond the covariates. Conclusions Cognitive, procedural, and communication assessments each added distinct incremental validity for written examination performance. This supports multidomain programmatic assessment design. The null result for theory assignments reveals a ceiling effect in immediate post-class recall testing, which has practical implications for assessment design in clinical courses. Delivering diverse assessment types through one platform offers a scalable way to generate the multisource evidence that programmatic assessment needs. </p>