Abstract
<p>Computing students say they care about AI ethics—but can they actually apply ethical reasoning when it matters? Most research on student AI competencies relies on self-reported surveys, a methodology vulnerable to the overconfidence it seeks to measure. This study introduces objective scenario-based testing. We surveyed 218 computing students (74% male) at a Polish university using a three-level framework that separates ethical attitudes (what students value), self-assessed knowledge (what they think they know), and objective knowledge (what they can actually demonstrate across five professionally grounded scenarios).The results reveal a substantial 29.5-percentage-point Knowledge-Calibration Gap: students rated their own ethical competence at 61% but scored only 31% on objective scenarios (paired dz = 0.89, p &lt; .001), with self-assessment accuracy effectively zero (ρ = .042). More revealing are the wrong-answer patterns—not random guessing but systematic misapplication of familiar heuristics: 47% of students confused learning strategies with verification processes, 43% chose investigation-oriented distractors over immediate harm cessation, and 35% defaulted to naive debiasing approaches the fairness literature has shown ineffective. Self-reported careful reviewers showed no advantage over the full sample (66.3% vs. 69.3% failure rate on the code-verification scenario), consistent with procedural miscalibration rather than deliberate misreporting.These findings challenge the adequacy of self-report research in AI education and demonstrate that the gap between ethical awareness and capability is not merely quantitative but qualitatively structured: students fail in predictable, heuristic-driven ways that can be diagnosed and targeted through scenario-based assessment embedded directly into computing curricula.</p>