LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data
arXiv cs.CL · 22 Oct 2026, 03:59 AEST
LongZero is a new method designed to enhance long-context reasoning in large language models without the need for external data. It allows models to synthesize their own questions and answers, improving their reasoning capabilities significantly.
Key facts
- Reinforcement learning (RL) is an effective way to strengthen the long-context reasoning of large language models (LLMs), but it needs training data that are long, diverse, challenging, and verifiable.
- Recently, RL from zero data, in which a model learns from tasks it proposes itself, has achieved excellent performance, but existing methods target short-context tasks.
- LongZero combines these self-written multi-hop reasoning tasks with self-written multi-round retrieval tasks and trains the model with GRPO.
- Experimental results show that LongZero can generalize to real tasks, and outperforms both baselines, suggesting a viable and promising path to improving long-context reasoning of LLMs.
Read aloud
LongZero is a new method designed to enhance long-context reasoning in large language models without the need for external data. It allows models to synthesize their own questions and answers, improving their reasoning capabilities significantly. Reinforcement learning RL is an effective way to strengthen the long-context reasoning of large language models L L M s, but it needs training data that are long, diverse, challenging, and verifiable. Recently, RL from zero data, in which a model learns from tasks it proposes itself, has achieved excellent performance, but existing methods target short-context tasks. LongZero combines these self-written multi-hop reasoning tasks with self-written multi-round retrieval tasks and trains the model with GRPO. Experimental results show that LongZero can generalize to real tasks, and outperforms both baselines, suggesting a viable and promising path to improving long-context reasoning of L L M s.
Sources in this cluster
- LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data
arXiv cs.CL · 22 Oct 2026, 03:59 AEST