You don't need long context
arXiv cs.AI · 4 Nov 2026, 04:17 AEST
Research indicates that longer context in language models may not be necessary for maintaining task accuracy. The study introduces semantic state space models (SSSMs) and highlights the efficiency of short contexts in various tasks.
Key facts
- Later tokens are costlier to generate than earlier tokens.
- On frontier open weight models, the disparity in cost may rise to as much as 7x.
- We show across a broad range of long-horizon and long-input tasks that frontier language models sacrifice little to no task accuracy when maintaining short contexts.
- LLM performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context.
Read aloud
Research indicates that longer context in language models may not be necessary for maintaining task accuracy. The study introduces semantic state space models SSSMs and highlights the efficiency of short contexts in various tasks. Later tokens are costlier to generate than earlier tokens. On frontier open weight models, the disparity in cost may rise to as much as 7x. We show across a broad range of long-horizon and long-input tasks that frontier language models sacrifice little to no task accuracy when maintaining short contexts. L L M performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context.
Sources in this cluster
- You don't need long context
arXiv cs.AI · 4 Nov 2026, 04:17 AEST