AI TrackerFrontier desk · AEST
Back to news
normalaitech

Beyond SAEs: Consistent Concept Discovery with Sparse Coding

arXiv cs.AI · 17 Oct 2026, 03:25 AEST

A new method called Consistent Sparse Dictionary Learning (ConSDL) improves the reliability of sparse autoencoders in concept discovery. It achieves significant gains in dictionary and coefficient consistency across various settings.

Key facts

Read aloud

A new method called Consistent Sparse Dictionary Learning ConSDL improves the reliability of sparse autoencoders in concept discovery. It achieves significant gains in dictionary and coefficient consistency across various settings. Sparse autoencoders SAEs explain foundation model representations by decomposing activations into sparse combinations of learned dictionary atoms. However, the learned dictionaries can vary across training runs, limiting the reliability and reproducibility of SAE-based analyses. Across sixteen synthetic settings, ConSDL achieves near-perfect dictionary and coefficient recovery. These gains translate to state-of-the-art cross-seed consistency across four vision and language settings, where ConSDL improves dictionary and coefficient consistency by seventeen point six percent and thirty five point nine percent, respectively, over the average of six SAE baselines.

Sources in this cluster