AI TrackerFrontier desk · AEST
Back to news
normalaimarkets

Finetuning with Sampling: SFT Learns Better Than You Think

arXiv cs.CL · 2 Oct 2026, 03:45 AEST

Introducing new capabilities to frontier models has long been the goal of posttraining, which predominantly employs supervised finetuning (SFT) and reinforcement learning (RL) to this end. Conventional wisdom dictates that RL enables strong generalization on new tasks without losing existing capabilities, while SFT is prone to weak generalization and catastrophic forgetting.

Key facts

Read aloud

Introducing new capabilities to frontier models has long been the goal of posttraining, which predominantly employs supervised finetuning SFT and reinforcement learning RL to this end. Conventional wisdom dictates that RL enables strong generalization on new tasks without losing existing capabilities, while SFT is prone to weak generalization and catastrophic forgetting. Introducing new capabilities to frontier models has long been the goal of posttraining, which predominantly employs supervised finetuning SFT and reinforcement learning RL to this end. Conventional wisdom dictates that RL enables strong generalization on new tasks without losing existing capabilities, while SFT is prone to weak generalization and catastrophic forgetting. At the same time, SFT can learn from off-policy expert data, whereas RL must rely on a model's ability to find successful trajectories with repeated sampling. In our work, we seek to leverage the strength of on-policy learning while utilizing the privileged information contained in off-policy data. However, rather than modifying the learning objective to accommodate this data, we instead tailor the data distribution to better suit the learner.

Sources in this cluster