AI TrackerFrontier desk · AEST
Back to news
normalaitech

Do "True" and "False" Add Up to One? Complement Consistency in LLM-Agent Opinion Simulations

arXiv cs.AI · 24 Oct 2026, 02:44 AEST

A study investigates the consistency of probability reports in LLM-agent opinion simulations. The findings reveal that complementary reports often do not sum to one, indicating a significant discrepancy in the models tested.

Key facts

Read aloud

A study investigates the consistency of probability reports in L L M-agent opinion simulations. The findings reveal that complementary reports often do not sum to one, indicating a significant discrepancy in the models tested. In L L M-agent opinion simulations, each agent reports the probability that a claim is true after seeing its own previous probability and the probabilities reported by its neighbors, and that number becomes its state in the next round. On two thousand three hundred four agent states taken from a running network simulation, Qwen's paired reports sum to one point one eight on average, and on forty eight real fact-checking claims from AVeriTeC they again exceed one. The discrepancy is two to ten times larger than the change produced by a same-meaning paraphrase, and it persists under repeated sampling and across peer group sizes and report strengths. Averaging each answer with the complement of its paired answer restores consistency by construction and lowers Brier loss averaged over the two wordings, but at the final round it beats the better of the two original wordings for only one of the three models.

Sources in this cluster