AI TrackerFrontier desk · AEST
Back to news
normalaitech

ProteinGeoBench: Frontier Structural-Biology Agents Are Fluent at Lookup but Lost in Geometry

arXiv cs.AI · 20 Oct 2026, 03:55 AEST

Frontier agents excel at retrieving structural metadata but struggle with geometric calculations. The study of ProteinGeoBench reveals significant variability in model performance based on evaluation configurations.

Key facts

Read aloud

Frontier agents excel at retrieving structural metadata but struggle with geometric calculations. The study of ProteinGeoBench reveals significant variability in model performance based on evaluation configurations. Frontier agents retrieve structural metadata well and compute geometry from coordinates less reliably. We study ProteinGeoBench, a suite of one hundred thirty one protein-structure tasks, using four thousand two hundred forty nine rollouts from four frontier models and four thousand four hundred eighty six further trials across twenty six models. Three of the four frontier models fall within zero point zero three one of one another zero point seven eight five to zero point eight one six with no difference between them resolved, while the fourth scores zero point seven zero seven and all three gaps to it exclude zero. Three of the four do better on metadata lookup than on numeric calculation. Numeric tasks are twenty eight of the one hundred thirty one but half of the twelve tasks that all four models fail.

Sources in this cluster