ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills
arXiv cs.CL · 9 Oct 2026, 03:44 AEST
ViSkill is a visual-native skill learning framework designed to enhance the performance of VLM agents. It achieves a success rate of 0.89, improving to 0.91 with cold-start initialization, outperforming existing baselines.
Key facts
- Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies.
- We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents.
- Retrieved skills guide both inference and reward shaping, while successful trajectories are distilled back into the library, forming a closed feedback loop in which skill accumulation and policy improvement reinforce each other.
- Evaluated on Sokoban, FrozenLake, and PrimitiveSkill, ViSkill achieves an overall success rate of 0.89, rising to 0.91 with cold-start initialization.
Read aloud
ViSkill is a visual-native skill learning framework designed to enhance the performance of VLM agents. It achieves a success rate of zero point eight nine, improving to zero point nine one with cold-start initialization, outperforming existing baselines. Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies. We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents. Retrieved skills guide both inference and reward shaping, while successful trajectories are distilled back into the library, forming a closed feedback loop in which skill accumulation and policy improvement reinforce each other. Evaluated on Sokoban, FrozenLake, and PrimitiveSkill, ViSkill achieves an overall success rate of zero point eight nine, rising to zero point nine one with cold-start initialization.
Sources in this cluster
- ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills
arXiv cs.CL · 9 Oct 2026, 03:44 AEST