OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
arXiv cs.AI · 20 Oct 2026, 03:59 AEST
OpenVLHarness introduces a multimodal reasoning harness designed to enhance visual reasoning capabilities in large language models. It improves performance on various benchmarks by integrating a capability layer and persistent multimodal memory.
Key facts
- Our controlled study on 22 benchmarks shows that simply increasing the inference budget of the base model yields only modest gains.
- The capability layer combines task-level tool interfaces with evidence rendering, providing consistent access to heterogeneous backends and converting their outputs into model-readable observations.
- Evaluated on 22 benchmarks with eight open and proprietary backbones, OpenVLHarness improves average scores by 9.7-12.6 points over matched no-tool baselines.
- With GPT-6 Luna at medium reasoning effort, OpenVLHarness outperforms the base model, Codex, and Visual Sketchpad at maximum effort by 7.7-8.5 points while incurring at most 37% of their model-token cost.
Read aloud
OpenVLHarness introduces a multimodal reasoning harness designed to enhance visual reasoning capabilities in large language models. It improves performance on various benchmarks by integrating a capability layer and persistent multimodal memory. Our controlled study on twenty two benchmarks shows that simply increasing the inference budget of the base model yields only modest gains. The capability layer combines task-level tool interfaces with evidence rendering, providing consistent access to heterogeneous backends and converting their outputs into model-readable observations. Evaluated on twenty two benchmarks with eight open and proprietary backbones, OpenVLHarness improves average scores by nine point seven-twelve point six points over matched no-tool baselines. With GPT-six Luna at medium reasoning effort, OpenVLHarness outperforms the base model, Codex, and Visual Sketchpad at maximum effort by seven point seven-eight point five points while incurring at most thirty seven percent of their model-token cost.
Sources in this cluster
- OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
arXiv cs.AI · 20 Oct 2026, 03:59 AEST