AI TrackerFrontier desk · AEST
Back to news
normalaitech

MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support

arXiv cs.CL · 16 Oct 2026, 03:51 AEST

MindEval2 introduces a new evaluation framework for AI chatbots used in mental health support. It aims to improve the assessment of these tools by providing more interpretable and actionable evaluation criteria.

Key facts

Read aloud

MindEval2 introduces a new evaluation framework for A I chatbots used in mental health support. It aims to improve the assessment of these tools by providing more interpretable and actionable evaluation criteria. With people increasingly turning to A I chatbots for mental health support, the need to evaluate these tools is becoming more pressing. To bridge these gaps, we introduce MindEval2, an evaluation framework that consists of two components: Mental Health Quality Rubrics MHQR, for evaluation, and MindSim, for patient simulation. For MHQR, we collaborate with PhD-level psychologists to develop a set of rubrics that break down response quality into explicit operational criteria that are more interpretable and actionable than existing evaluation taxonomies. We evaluate eighteen proprietary and open-weight models on MindEval2, finding that while frontier models have largely improved on issues related to surface level quality and boundaries, guidance calibration remains the dominant failure mode across all models.

Sources in this cluster

MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support · AI Tracker