Probing Spatial State Representations in Visual Foundation Models
arXiv cs.AI · 30 Oct 2026, 03:58 AEST
This study investigates how visual foundation models represent spatial states necessary for effective spatial perception. The findings reveal that while strong readout of visible scenes is present, it does not guarantee accurate object persistence or future-feature retrieval.
Key facts
- Spatial perception involves more than recognizing individual images: it calls for a persistent spatial state that is updated under self-motion and used to guide action.
- The resulting capability profiles vary across tasks, network layers, and prefix lengths: strong readout of the visible scene does not ensure accurate object persistence or future-feature retrieval.
- Native future prediction retains recoverable spatial structure, but a spatial reader trained on real features does not transfer to predicted features unchanged.
- Across the four levels, object persistence and actionability build on camera-pose readout, whereas feature prediction builds on visible-scene readout.
Read aloud
This study investigates how visual foundation models represent spatial states necessary for effective spatial perception. The findings reveal that while strong readout of visible scenes is present, it does not guarantee accurate object persistence or future-feature retrieval. Spatial perception involves more than recognizing individual images: it calls for a persistent spatial state that is updated under self-motion and used to guide action. The resulting capability profiles vary across tasks, network layers, and prefix lengths: strong readout of the visible scene does not ensure accurate object persistence or future-feature retrieval. Native future prediction retains recoverable spatial structure, but a spatial reader trained on real features does not transfer to predicted features unchanged. Across the four levels, object persistence and actionability build on camera-pose readout, whereas feature prediction builds on visible-scene readout.
Sources in this cluster
- Probing Spatial State Representations in Visual Foundation Models
arXiv cs.AI · 30 Oct 2026, 03:58 AEST