AI TrackerFrontier desk · AEST
Back to news
normalaitech

Same Simulation, Different Evidence: Where Retrieved Tool Output Costs LLM Agents Accuracy

arXiv cs.AI · 20 Oct 2026, 03:57 AEST

LLM agents can compute correct answers but still provide incorrect responses due to context management issues. The study highlights how retrieval methods impact the accuracy of models like GPT-5 and Claude when using a geochemistry simulator.

Key facts

Read aloud

L L M agents can compute correct answers but still provide incorrect responses due to context management issues. The study highlights how retrieval methods impact the accuracy of models like G P T five and Claude when using a geochemistry simulator. L L M agents that compute the right answer can still answer wrongly. When agents read the simulator's report through a section index rather than in context, G P T five.one and G P T five.two lose eleven point three and seven point nine points of accuracy. Under retrieval, they often skip the report, answer from their own higher-precision output files, and misround to a neighboring option, a failure we call evidence substitution. Naming the authoritative output raises accuracy by six point three-seven point nine points over plain retrieval for the three GPT models.

Sources in this cluster