Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
arXiv cs.AI · 27 Oct 2026, 03:44 AEST
A new robot policy called Zero-real Agent Policy (ZAP) has been developed to teach robots how to understand and accomplish tasks. ZAP achieves a 96.7% success rate on real-world tasks without using real-world robot data during training.
Key facts
- To achieve these two skills, we introduce Zero-real Agent Policy (ZAP), a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness.
- Without any real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra.
- We demonstrate that ZAP also generalizes to completely new objects, goals, strategies, and language by seating a test tube in a rack (and choosing a different hole in the rack when prompted), handing a test tube between arms mid-air, drawing an X on a tic-tac-toe board, and playing chess.
- Furthermore, ZAP also transfers to an unseen embodiment.
Read aloud
A new robot policy called Zero-real Agent Policy ZAP has been developed to teach robots how to understand and accomplish tasks. ZAP achieves a ninety six point seven percent success rate on real-world tasks without using real-world robot data during training. To achieve these two skills, we introduce Zero-real Agent Policy ZAP, a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness. Without any real-world robot data during training, ZAP reaches ninety six point seven percent success on real-world tasks, matching GPT-six Astra in our harness and outperforming the MolmoAct2 VLA thirty three point three percent, while running 3x faster and 17x cheaper than GPT-six Astra. We demonstrate that ZAP also generalizes to completely new objects, goals, strategies, and language by seating a test tube in a rack and choosing a different hole in the rack when prompted, handing a test tube between arms mid-air, drawing an X on a tic-tac-toe board, and playing chess. Furthermore, ZAP also transfers to an unseen embodiment.
Sources in this cluster
- Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
arXiv cs.AI · 27 Oct 2026, 03:44 AEST