AI TrackerFrontier desk · AEST
Back to news
normalaitech

PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios

arXiv cs.CL · 15 Oct 2026, 03:39 AEST

A new benchmark called PerSo evaluates the social intelligence of large language models in real-world persuasion scenarios. It assesses both cognitive and behavioral intelligence using dialogues from Reddit's ChangeMyView community.

Key facts

Read aloud

A new benchmark called PerSo evaluates the social intelligence of large language models in real-world persuasion scenarios. It assesses both cognitive and behavioral intelligence using dialogues from Reddit's ChangeMyView community. Understanding the social intelligence of large language models L L M s is increasingly important for applications involving human-A I interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an L L M-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation. Our evaluation of diverse L L M s shows that performance generally scales with model size but varies across dimensions. The strongest L L M slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.

Sources in this cluster