KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
arXiv cs.AI · 16 Oct 2026, 03:52 AEST
KnowGPO is a new reinforcement learning framework designed for deep search agents that focuses on task-relevant knowledge acquisition. It outperforms existing methods in experiments on various benchmarks.
Key facts
- Deep search agents solve knowledge-intensive tasks through long-horizon, multi-turn interactions, where assigning credit to individual search decisions is fundamentally difficult.
- We propose KnowGPO, a reinforcement learning framework that aligns trajectories by the task-relevant knowledge they have acquired rather than by their surface observations.
- Experiments on Xbench, BrowseComp-ZH, and GAIA show KnowGPO consistently outperforms outcome-only (GRPO) and process-supervision baselines (GiGPO, IGPO) by 3.8 to 4.2 points.
Read aloud
KnowGPO is a new reinforcement learning framework designed for deep search agents that focuses on task-relevant knowledge acquisition. It outperforms existing methods in experiments on various benchmarks. Deep search agents solve knowledge-intensive tasks through long-horizon, multi-turn interactions, where assigning credit to individual search decisions is fundamentally difficult. We propose KnowGPO, a reinforcement learning framework that aligns trajectories by the task-relevant knowledge they have acquired rather than by their surface observations. Experiments on Xbench, BrowseComp-ZH, and GAIA show KnowGPO consistently outperforms outcome-only GRPO and process-supervision baselines GiGPO, IGPO by three point eight to four point two points.
Sources in this cluster
- KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
arXiv cs.AI · 16 Oct 2026, 03:52 AEST