PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
arXiv cs.CL · 15 Oct 2026, 03:39 AEST
A new benchmark called PerSo evaluates the social intelligence of large language models in real-world persuasion scenarios. It assesses both cognitive and behavioral intelligence using dialogues from Reddit's ChangeMyView community.
Key facts
- Understanding the social intelligence of large language models (LLMs) is increasingly important for applications involving human-AI interaction and multi-agent environments.
- Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals.
- We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an LLM-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation.
- Our evaluation of diverse LLMs shows that performance generally scales with model size but varies across dimensions.
- The strongest LLM slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.
Read aloud
A new benchmark called PerSo evaluates the social intelligence of large language models in real-world persuasion scenarios. It assesses both cognitive and behavioral intelligence using dialogues from Reddit's ChangeMyView community. Understanding the social intelligence of large language models L L M s is increasingly important for applications involving human-A I interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an L L M-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation. Our evaluation of diverse L L M s shows that performance generally scales with model size but varies across dimensions. The strongest L L M slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.
Sources in this cluster
- PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
arXiv cs.CL · 15 Oct 2026, 03:39 AEST