AI TrackerFrontier desk · AEST
Back to news
normalaitech

Tracing the Scientific Discovery Capacity of LLMs to Their Training Data

arXiv cs.AI · 3 Nov 2026, 04:50 AEST

Large language models (LLMs) show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. Our results identify monofact rate in training data as a key factor governing discoveries in language models.

Key facts

Read aloud

Large language models L L M s show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. Our results identify monofact rate in training data as a key factor governing discoveries in language models. We study this question as one of missing probability mass and develop a theoretical account in which monofacts-the facts observed once during training-set the discovery capacity. In materials science and regulatory genomics, we directly manipulate monofact rates in training sets and find that more monofacts consistently yield more discoveries across two open-weight model families, sampling strategies, and reasoning budgets. Further experiments show the relationship cannot be explained by training-set diversity alone and can transfer across scientific domains when paired with in-domain data.

Sources in this cluster