Abstract
<title>Abstract</title> <p>Learning efficient and task-relevant representations from high-dimensional image observations is a fundamental challenge in AI-era computational imaging, visual reinforcement learning, and image-based decision-making systems. Although model-based reinforcement learning has shown strong potential for improving sample efficiency, existing world models commonly rely on pixel-level reconstruction objectives, which may allocate substantial model capacity to background textures and other task-irrelevant visual details. This limitation can weaken the learned visual representations, reduce coding efficiency in latent world models, and degrade downstream control. To address this problem, we propose Explain-As-You-Learn, a self-explanation-guided world modeling framework for task-relevant visual representation learning. The proposed Self-Explanation-Guided World Model (SEGWM) transforms visual explanations from post-hoc interpretability tools into active learning signals that guide the representation learning process. Specifically, SEGWM first derives raw visual saliency maps from the gradients of the policy output to identify image regions that are relevant to action selection. To improve the robustness and semantic coherence of these explanations, we further introduce a teacher-student segmentation pipeline that aggregates noisy gradient-based maps into coherent object-level explanation maps. These maps are then used as supervisory targets for an auxiliary explanation predictor, forming a self-supervised learning loop in which more informative explanations promote more task-relevant visual representations, and improved representations in turn produce clearer explanations. Extensive experiments on the Atari 100K benchmark demonstrate that SEGWM improves sample efficiency and achieves competitive or superior final performance compared with strong world-model baselines. Ablation studies and qualitative analyses further show that the proposed explanation-guided mechanism enhances both representation quality and model interpretability. These results suggest a promising direction for building efficient, interpretable, and task-aware visual representation learning systems for computational imaging and image-based reinforcement learning.</p>