Abstract
<title>Abstract</title> <p>Programming practice is a progressive learning process in which learners improve their skills through repeated submissions and gradual progress toward correctness. In online judge systems, early prediction of problem completion is important for understanding learner progress during problem solving. However, many existing approaches rely mainly on final binary outcomes, which may not adequately capture intermediate progress and learners’ evolving capability. To address this gap, this study proposes an early success prediction framework that evaluates three formulations: binary outcome, ordinal correctness, and a proposed partial-correctness and progress-aware formulation. Using Aizu Online Judge (AOJ) submission logs, the prediction task determines whether a learner will solve a problem within the next K attempts based on early submission behavior. Experimental results demonstrate that the proposed formulation consistently outperforms binary-outcome and ordinal correctness formulations across six machine learning models and evaluation metrics. On the Algorithm and Data Structures I (ALDS1) test set, random forest achieved the best performance under this formulation, showing relative improvements in PR-AUC and accuracy by 6.20% and 6.63%, respectively, over the binary-outcome formulation, and by 5.87% and 6.78%, respectively, over the ordinal-correctness formulation. Additional AOJ course evaluation and K sensitivity analysis further supported the stability of the formulation. Risk-based analysis also showed clearer separation among the three risk groups than the other two formulations, with low-risk groups showing more favorable attempt-wise success patterns than high-risk groups. Overall, the proposed formulation more effectively captures learning progression and provides an interpretable basis for early identification and instructional support in programming problem-solving environments.</p>