SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification
arXiv cs.CL · 5 Nov 2026, 04:34 AEST
SpecCompact is a new framework designed to accelerate context compaction for LLM agents by using slot-parallel semantic speculation. It achieves significant speedups in compaction while balancing information retention and latency.
Key facts
- LLM agents use context compaction to manage growing interaction histories, but summarizing long histories with large models can introduce substantial delays.
- We present SpecCompact, a framework that accelerates compaction through slot-parallel semantic speculation.
- Across AppWorld and $τ^3$ benchmarks with two target models, SpecCompact achieves $2.04$--$5.63 imes$ compaction speedups over Hermes-style summarization.
- A user study within Hermes further shows an $8.68 imes$ median compaction speedup.
Read aloud
SpecCompact is a new framework designed to accelerate context compaction for L L M agents by using slot-parallel semantic speculation. It achieves significant speedups in compaction while balancing information retention and latency. L L M agents use context compaction to manage growing interaction histories, but summarizing long histories with large models can introduce substantial delays. We present SpecCompact, a framework that accelerates compaction through slot-parallel semantic speculation. Across AppWorld and $τ^three$ benchmarks with two target models, SpecCompact achieves two dollars and four cents$--five dollars and sixty three cents imes$ compaction speedups over Hermes-style summarization. A user study within Hermes further shows an eight dollars and sixty eight cents imes$ median compaction speedup.
Sources in this cluster
- SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification
arXiv cs.CL · 5 Nov 2026, 04:34 AEST