AI TrackerFrontier desk · AEST
Back to news
normalaitech

Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy

arXiv cs.AI · 27 Oct 2026, 03:51 AEST

This report evaluates the capabilities of GPT-6-Astra as an agent controlling a robot using EBench. It compares direct control and harness with policy configurations, highlighting performance variations across different task types.

Key facts

Read aloud

This report evaluates the capabilities of GPT-six-Astra as an agent controlling a robot using EBench. It compares direct control and harness with policy configurations, highlighting performance variations across different task types. Frontier multimodal large language models MLLMs can control robots without robot-specific training, but how their capabilities compare and interact with post-trained vision-language-action VLA and world action model WAM policies across manipulation demands remains to be thoroughly evaluated. Under direct control, the agent achieves competitive overall performance relative to post-trained policies. In the harness with policy, invoking a post-trained VLA or WAM combines the complementary strengths of both components, raising overall task success to the level of the strongest compared post-trained policies while substantially reducing token cost. However, performance remains below the per-task upper bound of the standalone components due to suboptimal policy invocation, fixed-instruction execution without subgoals, and grasp disruption when the agent resumes control.

Sources in this cluster