AI TrackerFrontier desk · AEST
Back to news
normalaitech

Rephrase Before You Act: Characterizing and Mitigating Language Sensitivity in Vision-Language-Action Models

arXiv cs.CL · 8 Oct 2026, 03:57 AEST

Vision-language-action models (VLAs) are highly sensitive to instruction phrasing, lacking the robustness of their vision-language counterparts. A systematic approach to rephrasing can significantly improve performance on various tasks without the need for retraining.

Key facts

Read aloud

Vision-language-action models VLAs are highly sensitive to instruction phrasing, lacking the robustness of their vision-language counterparts. A systematic approach to rephrasing can significantly improve performance on various tasks without the need for retraining. Vision-language-action models VLAs are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-language models they are built on. A one-word edit can move success by tens of points: $π zero point five $ turns on a LIBERO stove one hundred percent of the time for 'switch on the stove' and two percent for 'switch on the hot plate'. We characterize this sensitivity with statistically tested single-edit swings and an oracle phrase search, which shows that phrasing alone nearly closes the twenty one-point gap between in-distribution and out-of-distribution tasks. The rules improve the frozen $π 0$ by sixteen to twenty seven percent relative on twelve held-out tasks across adversarial, VLM-generated, and human-generated phrasings, with gains concentrated on out-of-distribution tasks. The method requires no retraining and no per-step verification, and applies zero-shot to unseen tasks and instructions.

Sources in this cluster