Abstract
<title>Abstract</title> <p>We study the problem of modeling learner knowledge in digital environments where complex tasks are solved through iterative attempts supported by feedback. In such settings, standard Elo Rating Systems (ERS) treat all successful outcomes equivalently, failing to distinguish independent success from success achieved with assistance, which may lead to biased knowledge estimates. To address this limitation, we propose two feedback-aware extensions of ERS. The New Item (NI) model represents each (item, feedback level) pair as a distinct task, while the Probability Modifier (PM) model distinguishes between assisted and unassisted attempts and models feedback as a level-dependent shift in the probability of success within a Rasch-based formulation. These approaches reflect complementary assumptions about the role of feedback. We evaluate the models on 47,224 learner–item interactions from 296 university learners engaged in a complex iterative task. Results show that both NI and PM significantly outperform feedback-agnostic ERS models across multiple complementary evaluation settings, including robustness to learner-population variations, consistency across learners, and generalization to unseen learners, while preserving meaningful knowledge discrimination and stable difficulty estimates. These findings highlight the importance of explicitly modeling feedback in iterative learning contexts.</p>