AI TrackerFrontier desk · AEST
Back to news
normalaitech

Latent Compaction with Test-Time Training

arXiv cs.CL · 15 Oct 2026, 03:57 AEST

Latent Compaction introduces a method for compressing key-value caches during generation while preserving information for unseen questions. Experiments show significant improvements in accuracy and retrieval gains across different model scales and families.

Key facts

Read aloud

Latent Compaction introduces a method for compressing key-value caches during generation while preserving information for unseen questions. Experiments show significant improvements in accuracy and retrieval gains across different model scales and families. Long contexts enlarge key--value KV caches, increasing memory use and attention costs during generation. We introduce Latent Compaction, which learns a compact document-specific cache through test-time training. Experiments on RULER, LongBench, and Phonebook Lookup demonstrate substantial gains over budget-matched compression baselines. On a RULER, for example, our method achieves ninety six point seven percent accuracy at thirty two× compression, compared with forty point zero percent for KVZip+.

Sources in this cluster