Latent Compaction with Test-Time Training
arXiv cs.CL · 15 Oct 2026, 03:57 AEST
Latent Compaction introduces a method for compressing key-value caches during generation while preserving information for unseen questions. Experiments show significant improvements in accuracy and retrieval gains across different model scales and families.
Key facts
- Long contexts enlarge key--value (KV) caches, increasing memory use and attention costs during generation.
- We introduce Latent Compaction, which learns a compact document-specific cache through test-time training.
- Experiments on RULER, LongBench, and Phonebook Lookup demonstrate substantial gains over budget-matched compression baselines.
- On a RULER, for example, our method achieves 96.7% accuracy at 32× compression, compared with 40.0% for KVZip+.
Read aloud
Latent Compaction introduces a method for compressing key-value caches during generation while preserving information for unseen questions. Experiments show significant improvements in accuracy and retrieval gains across different model scales and families. Long contexts enlarge key--value KV caches, increasing memory use and attention costs during generation. We introduce Latent Compaction, which learns a compact document-specific cache through test-time training. Experiments on RULER, LongBench, and Phonebook Lookup demonstrate substantial gains over budget-matched compression baselines. On a RULER, for example, our method achieves ninety six point seven percent accuracy at thirty two× compression, compared with forty point zero percent for KVZip+.
Sources in this cluster
- Latent Compaction with Test-Time Training
arXiv cs.CL · 15 Oct 2026, 03:57 AEST