FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?
arXiv cs.CL · 9 Oct 2026, 03:55 AEST
FastBench introduces a new benchmark for evaluating high-dynamic perception in streaming video large language models (VLMs). The study highlights significant limitations in current models' ability to process fast events in real-world video streams.
Key facts
- Streaming Video Large Language Models (VLMs) enable continuous video understanding, yet existing benchmarks focus on low-dynamic scenarios.
- FastBench contains 306 QA pairs across eight domains, six capabilities, and forward, instant, and backward temporal scopes, with human-annotated evidence intervals.
- Experiments reveal substantial limitations: the strongest model, Gemini-3.5-Flash, scores only 50.7%.
- ProactiveFrame outperforms sparse uniform sampling by 5.4 and 1.5 percentage points, yet remains well below oracle-guided focusing.
Read aloud
FastBench introduces a new benchmark for evaluating high-dynamic perception in streaming video large language models VLMs. The study highlights significant limitations in current models' ability to process fast events in real-world video streams. Streaming Video Large Language Models VLMs enable continuous video understanding, yet existing benchmarks focus on low-dynamic scenarios. FastBench contains three hundred six QA pairs across eight domains, six capabilities, and forward, instant, and backward temporal scopes, with human-annotated evidence intervals. Experiments reveal substantial limitations: the strongest model, Gemini-three point five-Flash, scores only fifty point seven percent. ProactiveFrame outperforms sparse uniform sampling by five point four and one point five percentage points, yet remains well below oracle-guided focusing.
Sources in this cluster
- FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?
arXiv cs.CL · 9 Oct 2026, 03:55 AEST