AI TrackerFrontier desk · AEST
Back to news
normalaitech

Towards interpreting any residual stream direction by scaling activation inversion

arXiv cs.AI · 6 Nov 2026, 04:54 AEST

A new approach called Maximally-Activating Example Metamodels (MAEMs) aims to interpret directions in a language model's residual stream. This method shows improved performance over traditional corpus search and natural-language autoencoders in generating informative text exemplars.

Key facts

Read aloud

A new approach called Maximally-Activating Example Metamodels MAEMs aims to interpret directions in a language model's residual stream. This method shows improved performance over traditional corpus search and natural-language autoencoders in generating informative text exemplars. To interpret a direction in a language model's residual stream, we can look for text that strongly activates it. We develop Maximally-Activating Example Metamodels MAEMs, language models which map residual stream directions from a subject model to text exemplars that strongly activate those directions. Empirically, we find the resultant metamodel produces informative exemplars: a Qwen3.six-27B MAEM inverts held-out activations better than corpus search or a natural-language autoencoder NLA. It also identifies concepts behind steering vectors about as well as NLAs, recovers the payloads of LoRA backdoors more often than corpus search, and can even verbalise some model workspace inner monologue.

Sources in this cluster

Towards interpreting any residual stream direction by scaling activation inversion · AI Tracker