AI TrackerFrontier desk · AEST
Back to news
normalaitech

Validity Without Ground Truth: What Stated-Preference Economics Offers the Evaluation of Language Models

arXiv cs.AI · 8 Oct 2026, 03:51 AEST

Stated-preference economics offers a framework for evaluating language models in the absence of correct answers. This approach assesses validity through concepts like reliability and incentive compatibility, demonstrating its effectiveness with a water-quality valuation survey.

Key facts

Read aloud

Stated-preference economics offers a framework for evaluating language models in the absence of correct answers. This approach assesses validity through concepts like reliability and incentive compatibility, demonstrating its effectiveness with a water-quality valuation survey. Many of the questions now put to large language models have no correct answer to score against: what a policy is worth, which option a user should choose, how to weigh competing values. Stated-preference economics has faced this problem for decades. We demonstrate the approach using a published water-quality stated preference economic valuation survey Vossler et al. twenty twenty three administered to six models. Two older models fail the most basic test at a household income level of seventy five thousand dollars and the two newest pass every test of theoretical validity we can score, but diverge on convergent validity.

Sources in this cluster

Validity Without Ground Truth: What Stated-Preference Economics Offers the Evaluation of Language Models · AI Tracker