AI TrackerFrontier desk · AEST
Back to news
normalaitech

MedPhysBench: A Benchmark for Evaluating Language Models on Medical Physics Calculations and Error Detection

arXiv cs.AI · 4 Nov 2026, 04:40 AEST

MedPhysBench is a benchmark designed to evaluate language models on medical physics calculations and error detection. It includes ten seeded generators of calculation items across various medical physics domains and assesses model performance on both calculations and error detection.

Key facts

Read aloud

MedPhysBench is a benchmark designed to evaluate language models on medical physics calculations and error detection. It includes ten seeded generators of calculation items across various medical physics domains and assesses model performance on both calculations and error detection. Medical physicists turn a prescription into a delivered dose and check one another's calculations, because one error in a beam calibration or a monitor-unit MU calculation can affect every patient treated with it. On a held-out test split of one thousand two hundred items half for two models, twelve models scored thirty three point three to one hundred point zero percent on calculation. Five models detected at least ninety six point seven percent of mostly large planted errors with at most four percent false alarms; for four of the other seven, checking only the result raised false alarms significantly. In an exploratory targeted test with twenty public cases of an omitted inverse-square correction, the three best models tested caught it in every worksheet.

Sources in this cluster