AI TrackerFrontier desk · AEST
Back to news
normalaitech

Capable Teacher, Myopic Scorer: Reward-Free Teacher Repair for Agentic On-Policy Distillation

arXiv cs.CL · 21 Oct 2026, 03:40 AEST

A new method called Rollout-to-Scorer Distillation (R2S) aims to improve agentic on-policy distillation by addressing the limitations of traditional scoring methods. R2S repairs the teacher's scoring process to enhance the training of student agents without relying on task rewards.

Key facts

Read aloud

A new method called Rollout-to-Scorer Distillation R2S aims to improve agentic on-policy distillation by addressing the limitations of traditional scoring methods. R2S repairs the teacher's scoring process to enhance the training of student agents without relying on task rewards. Agentic on-policy distillation OPD trains a student agent on its own trajectories while a teacher scores every token. We attribute this to the myopia of next-token scoring. We propose Rollout-to-Scorer Distillation R2S, which repairs the scorer before distillation by mining high-disagreement states from the untrained student. Across three environments, two teachers and four students, R2S outperforms three state-of-the-art agentic-OPD methods in all twelve teacher-student-environment cells.

Sources in this cluster

Capable Teacher, Myopic Scorer: Reward-Free Teacher Repair for Agentic On-Policy Distillation · AI Tracker