Quantifying Claim Smoothing by LLM Research Agents
arXiv cs.CL · 27 Oct 2026, 03:29 AEST
Research shows that large language models (LLMs) often produce overly confident syntheses that misrepresent the underlying documents. This phenomenon, termed 'smoothing', leads to conclusions that are more decisive than the supporting evidence allows.
Key facts
- In this work, we find that LLMs produce long-form research reports that read as confident syntheses but which fail to accurately reflect the underlying documents.
- We call this failure smoothing.
- In collapsing diverse and potentially conflicting sources into a single narrative, the model discards the evidential 'jaggedness' of its corpus and asserts conclusions more decisive than the citations can support.
- We introduce a claim level scoring pipeline, SmoothScore, to quantify this effect.
- Across 7 systems and 4 datasets, we find that agents consistently produce stronger claims than warranted.
Read aloud
Research shows that large language models L L M s often produce overly confident syntheses that misrepresent the underlying documents. This phenomenon, termed 'smoothing', leads to conclusions that are more decisive than the supporting evidence allows. In this work, we find that L L M s produce long-form research reports that read as confident syntheses but which fail to accurately reflect the underlying documents. We call this failure smoothing. In collapsing diverse and potentially conflicting sources into a single narrative, the model discards the evidential 'jaggedness' of its corpus and asserts conclusions more decisive than the citations can support. We introduce a claim level scoring pipeline, SmoothScore, to quantify this effect. Across seven systems and four datasets, we find that agents consistently produce stronger claims than warranted.
Sources in this cluster
- Quantifying Claim Smoothing by LLM Research Agents
arXiv cs.CL · 27 Oct 2026, 03:29 AEST