AI TrackerFrontier desk · AEST
Back to news
normalaitech

NgramRL: Accelerating RL Post-Training for Structured Generation via Rollout-Aware N-gram Speculative Decoding

arXiv cs.AI · 24 Oct 2026, 02:47 AEST

NgramRL is a new method designed to enhance reinforcement learning post-training for large language models by improving rollout generation efficiency. It utilizes a per-prompt n-gram drafter that leverages recent rollout history without requiring additional model training or GPU resources.

Key facts

Read aloud

NgramRL is a new method designed to enhance reinforcement learning post-training for large language models by improving rollout generation efficiency. It utilizes a per-prompt n-gram drafter that leverages recent rollout history without requiring additional model training or G P U resources. Reinforcement learning RL post-training improves the reasoning capabilities of large language models, but its cost is dominated by autoregressive rollout generation, which can exceed seventy percent of training time. We propose NgramRL, a per-prompt n-gram drafter constructed from bounded recent rollout history. NgramRL requires no auxiliary model, drafter training, or persistent G P U-resident drafter state, while target-model verification preserves the rollout distribution. Across two model families and three reasoning and structured-generation workloads, NgramRL commits three point six zero-seven point seven seven tokens per target-model forward in the warm-cache regime, up to one.99x more than SuffixDecoding and seven.0x more than prompt-lookup decoding.

Sources in this cluster

NgramRL: Accelerating RL Post-Training for Structured Generation via Rollout-Aware N-gram Speculative Decoding · AI Tracker