OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding
arXiv cs.CL · 14 Oct 2026, 03:54 AEST
OmniCompass introduces a holistic benchmark for evaluating omni-modal video grounding and understanding. It includes two tracks assessing vision-language and audio-visual models across various tasks.
Key facts
- OmniCompass defines two evaluation tracks: OmniCompass-VL assesses vision-language models on tasks requiring visual evidence, while OmniCompass-AV assesses omni-modal models on tasks requiring joint audio-visual evidence.
- Each track includes three complementary tasks: temporal grounding (TG), spatial grounding (SG), and question answering (QA), assessing when a referenced target appears, where it is, and what happens.
- OmniCompass-VL contains 820 videos and 2,460 instances across domains and three duration types: short, medium, and long.
- Evaluations of 50 MLLMs show that open-source models still lag behind proprietary counterparts, particularly in audio-visual understanding.
Read aloud
OmniCompass introduces a holistic benchmark for evaluating omni-modal video grounding and understanding. It includes two tracks assessing vision-language and audio-visual models across various tasks. OmniCompass defines two evaluation tracks: OmniCompass-VL assesses vision-language models on tasks requiring visual evidence, while OmniCompass-AV assesses omni-modal models on tasks requiring joint audio-visual evidence. Each track includes three complementary tasks: temporal grounding TG, spatial grounding SG, and question answering QA, assessing when a referenced target appears, where it is, and what happens. OmniCompass-VL contains eight hundred twenty videos and two thousand four hundred sixty instances across domains and three duration types: short, medium, and long. Evaluations of fifty MLLMs show that open-source models still lag behind proprietary counterparts, particularly in audio-visual understanding.
Sources in this cluster
- OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding
arXiv cs.CL · 14 Oct 2026, 03:54 AEST