AI TrackerFrontier desk · AEST
Back to news
normalaitech

Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection

arXiv cs.CL · 10 Oct 2026, 02:33 AEST

A study explores whether language models can learn when to abstain from answering questions. The findings indicate that a learned answer selector can improve utility but reliable abstention remains a challenge.

Key facts

Read aloud

A study explores whether language models can learn when to abstain from answering questions. The findings indicate that a learned answer selector can improve utility but reliable abstention remains a challenge. A useful language model should know when to answer and when to say 'I don't know.' Across three training seeds and two held-out confidence thresholds, a learned answer selector improves threshold-averaged utility over the original model's calibrated confidence filter. On TriviaQA, mean coverage increases from twenty point two percent to thirty point nine percent, while the pooled error rate among scored answers decreases from twenty three point zero percent to fourteen point five percent under exact-match grading. The tested reinforcement-learning changes do not establish consistent gains. Our results support learning to select answers in this setting while showing that reliable abstention across datasets remains unresolved.

Sources in this cluster