AI TrackerFrontier desk · AEST
Back to news
normalaitech

Reinforcement Learning with Visually Attributable On-Policy Self-Distillation

arXiv cs.AI · 20 Oct 2026, 03:59 AEST

A new approach called reinforcement learning with visually attributable on-policy self-distillation aims to improve visual reasoning in reinforcement learning. This method addresses issues of error localization and contamination in token-level corrections by grounding updates in visual evidence.

Key facts

Read aloud

A new approach called reinforcement learning with visually attributable on-policy self-distillation aims to improve visual reasoning in reinforcement learning. This method addresses issues of error localization and contamination in token-level corrections by grounding updates in visual evidence. Multimodal reinforcement learning with verifiable rewards RLVR provides reliable task-level supervision, yet offers limited guidance for localizing errors in visual reasoning. On-policy self-distillation OPSD complements this supervision with dense token-level corrections from a privileged teacher equipped with an evidence-centered visual view. We formalize this phenomenon as privileged-correction contamination: a token-level correction is contaminated when its update direction cannot be consistently attributed to the privileged visual evidence. To address this issue, we propose reinforcement learning with visually attributable on-policy self-distillation, an evidence-consistent paradigm that augments task-level RLVR supervision with privileged token-level corrections grounded in visual evidence. Experiments across six visual reasoning benchmarks show that VAG consistently outperforms GRPO and OPSD methods, establishing new state-of-the-art average performance.

Sources in this cluster