Asymmetric On-Policy Distillation Learns In-Context Exploration
arXiv cs.AI · 22 Oct 2026, 03:50 AEST
A new framework called ASTEROID has been developed to improve exploration in sequential decision-making under partial observability. It allows a student agent to learn efficient exploration behaviors by leveraging on-policy contexts and expert action labels.
Key facts
- Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively.
- In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration.
- Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration.
- Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation.
Read aloud
A new framework called ASTEROID has been developed to improve exploration in sequential decision-making under partial observability. It allows a student agent to learn efficient exploration behaviors by leveraging on-policy contexts and expert action labels. Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +forty percent over standard distillation.
Sources in this cluster
- Asymmetric On-Policy Distillation Learns In-Context Exploration
arXiv cs.AI · 22 Oct 2026, 03:50 AEST