AI TrackerFrontier desk · AEST
Back to news
normalaitech

TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts

arXiv cs.AI · 6 Oct 2026, 03:57 AEST

TasteVal is a new benchmark designed to evaluate the experimental research taste of AI models. It measures how effectively models can design experiments and interpret results compared to human experts.

Key facts

Read aloud

TasteVal is a new benchmark designed to evaluate the experimental research taste of A I models. It measures how effectively models can design experiments and interpret results compared to human experts. We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models. TasteVal measures the experimental component of research taste; given a fixed research problem, we measure how well a model iteratively designs experiments and draws conclusions from their outcomes. The best-performing model, Opus five point five, exceeds our expert baseline, with a compute multiplier of two.3x ninety five percent CI one point one five-four point three seven, at roughly one/thirty of our baseliners' average per-run cost. On TasteVal, the compute multiplier of frontier models has doubled approximately every three point zero months since December twenty twenty five ninety five percent CI one point seven-five point zero, up from every fourteen months between twenty twenty three and December twenty twenty five.

Sources in this cluster