LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models
arXiv cs.AI · 14 Oct 2026, 03:57 AEST
LIBERO-RIG is a benchmark designed to test the robustness of vision-language-action models against hardware variations. It evaluates nine models across 210 conditions, revealing significant performance degradation under different perturbation axes.
Key facts
- Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, calibrations drift, controllers run at different rates, and end-effector pads wear down.
- The benchmark spans eleven perturbation axes, including camera mounting and optics, camera-to-arm calibration, actuation gain, control rate, proprioceptive noise, base pose, table height, and gripper pads.
- Pooled success falls from 0.932 to 0.767 at the middle severity and 0.509 at the highest severity.
- Performance degradation varies substantially across perturbation axes and action interfaces, and training on visual perturbations does not mitigate faults that leave camera images unchanged.
Read aloud
LIBERO-RIG is a benchmark designed to test the robustness of vision-language-action models against hardware variations. It evaluates nine models across two hundred ten conditions, revealing significant performance degradation under different perturbation axes. Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, calibrations drift, controllers run at different rates, and end-effector pads wear down. The benchmark spans eleven perturbation axes, including camera mounting and optics, camera-to-arm calibration, actuation gain, control rate, proprioceptive noise, base pose, table height, and gripper pads. Pooled success falls from zero point nine three two to zero point seven six seven at the middle severity and zero point five zero nine at the highest severity. Performance degradation varies substantially across perturbation axes and action interfaces, and training on visual perturbations does not mitigate faults that leave camera images unchanged.
Sources in this cluster
- LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models
arXiv cs.AI · 14 Oct 2026, 03:57 AEST