AI TrackerFrontier desk · AEST
Back to news
normalaitech

VocalGrad: Evaluating Acoustic Perception in Audio Language Models

arXiv cs.CL · 17 Oct 2026, 02:54 AEST

Recent research introduces VocalGrad, a benchmark for evaluating the acoustic perception capabilities of large audio language models (LALMs). The findings indicate a significant gap in performance between human annotators and LALMs in recognizing low-level acoustic variations.

Key facts

Read aloud

Recent research introduces VocalGrad, a benchmark for evaluating the acoustic perception capabilities of large audio language models LALMs. The findings indicate a significant gap in performance between human annotators and LALMs in recognizing low-level acoustic variations. Recent large audio language models LALMs have extended their capabilities beyond lexical speech understanding to tasks requiring paralinguistic and non-linguistic audio understanding, such as emotion recognition. To address this issue, we introduce VocalGrad, a benchmark designed to evaluate LALMs' ability to perceive low-level acoustic variations. Our benchmark results reveal a substantial gap in acoustic perception between human annotators and state-of-the-art LALMs: while human annotators achieve high accuracy, LALMs perform at near-chance levels. Our analysis reveals two distinct failure modes: for some categories, task-relevant information is preserved in internal representations but not utilized by the language model head; for others, the information becomes progressively less linearly decodable across model layers.

Sources in this cluster