AI TrackerFrontier desk · AEST
Back to news
normalaitech

On the Reliability of Majority-Vote Pseudo-Labels in Test-Time Reinforcement Learning

arXiv cs.AI · 7 Nov 2026, 04:49 AEST

The study examines the reliability of majority-vote pseudo-labels in test-time reinforcement learning. It introduces a new method, Prompt-Conditional Distributionally Robust DPO, to improve the adaptation process.

Key facts

Read aloud

The study examines the reliability of majority-vote pseudo-labels in test-time reinforcement learning. It introduces a new method, Prompt-Conditional Distributionally Robust DPO, to improve the adaptation process. Test-time reinforcement learning TTRL adapts pretrained language models to unlabeled target-domain data using self-generated pseudo-labels, whose reliability remains poorly understood. We analyze majority-vote supervision for TTRL with online Direct Preference Optimization DPO and decompose its error into population-majority bias and prompt-dependent finite-rollout variation. We propose Prompt-Conditional Distributionally Robust DPO PC-Dr.DPO, which conditions within-prompt robust reweighting on observed vote reliability to make updates from sampling-sensitive prompts more conservative. Across six mathematical and knowledge-intensive reasoning benchmarks and five model backbones, PC-Dr.DPO matches or outperforms the strongest TTRL baseline in twenty four of thirty settings.

Sources in this cluster