TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts
arXiv cs.AI · 6 Oct 2026, 03:57 AEST
TasteVal is a new benchmark designed to evaluate the experimental research taste of AI models. It measures how effectively models can design experiments and interpret results compared to human experts.
Key facts
- We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models.
- TasteVal measures the experimental component of research taste; given a fixed research problem, we measure how well a model iteratively designs experiments and draws conclusions from their outcomes.
- The best-performing model, Opus 5.5, exceeds our expert baseline, with a compute multiplier of 2.3x (95% CI 1.15-4.37), at roughly 1/30 of our baseliners' average per-run cost.
- On TasteVal, the compute multiplier of frontier models has doubled approximately every 3.0 months since December 2025 (95% CI 1.7-5.0), up from every 14 months between 2023 and December 2025.
Read aloud
TasteVal is a new benchmark designed to evaluate the experimental research taste of A I models. It measures how effectively models can design experiments and interpret results compared to human experts. We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models. TasteVal measures the experimental component of research taste; given a fixed research problem, we measure how well a model iteratively designs experiments and draws conclusions from their outcomes. The best-performing model, Opus five point five, exceeds our expert baseline, with a compute multiplier of two.3x ninety five percent CI one point one five-four point three seven, at roughly one/thirty of our baseliners' average per-run cost. On TasteVal, the compute multiplier of frontier models has doubled approximately every three point zero months since December twenty twenty five ninety five percent CI one point seven-five point zero, up from every fourteen months between twenty twenty three and December twenty twenty five.
Sources in this cluster
- TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts
arXiv cs.AI · 6 Oct 2026, 03:57 AEST