AI TrackerFrontier desk · AEST
Back to news
normalaitech

Learning to Read the Contextual Tokens in Diffusion Transformers

arXiv cs.AI · 6 Oct 2026, 03:59 AEST

Multimodal Diffusion Transformers (MM-DiTs) process visual and textual representations, updating text tokens through multimodal attention. This study introduces a framework for interpreting contextual tokens, revealing their role in encoding rich, global representations of emerging scenes.

Key facts

Read aloud

Multimodal Diffusion Transformers MM-DiTs process visual and textual representations, updating text tokens through multimodal attention. This study introduces a framework for interpreting contextual tokens, revealing their role in encoding rich, global representations of emerging scenes. These models repeatedly update the text tokens through multimodal attention, forming dynamic contextual tokens whose function is not well understood. We train a lightweight bottleneck network that maps intermediate contextual tokens into the input space of a frozen Large Language Model L L M, allowing the L L M to answer questions about the emerging image directly from these hidden representations. Our reader reveals that contextual tokens encode a rich, global representation of the emerging scene: generation-specific semantics, including attributes left underspecified by the prompt, are accessible surprisingly early in denoising. Remarkably, this information remains decodable even when the MM-DiT receives an empty prompt, showing that contextual tokens accumulate substantial image-specific information from the evolving visual representation itself. Together, our results establish contextual tokens as both an interpretable view into the internal dynamics of MM-DiTs and an effective target for improving generative models.

Sources in this cluster

Learning to Read the Contextual Tokens in Diffusion Transformers · AI Tracker