AI TrackerFrontier desk · AEST
Back to news
normalaitech

When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors

arXiv cs.CL · 10 Oct 2026, 03:50 AEST

Vision-Language Models (VLMs) struggle to accurately represent spatial relations compared to lexical information. A new metric and benchmark reveal that these models lose relational information when aggregating input into global representations.

Key facts

Read aloud

Vision-Language Models VLMs struggle to accurately represent spatial relations compared to lexical information. A new metric and benchmark reveal that these models lose relational information when aggregating input into global representations. We introduce EICS Evaluation-Invariant Concept Stability, a metric for testing whether representations consistently encode concepts under controlled one-concept interventions. Across multiple VLM architectures, we find that spatial relations are represented substantially less faithfully than lexical information, even when the same models score well above chance on compositional benchmarks. Further analysis shows that the relation is identifiable at the predicate's own tokens but is lost when the sentence is aggregated into a global representation, whether through the EOS token or by mean pooling.

Sources in this cluster