AI TrackerFrontier desk · AEST
Back to news
normalaitech

Adaptive Multi-Token Prediction: How far should we go?

arXiv cs.AI · 21 Oct 2026, 03:56 AEST

Adaptive Multi-Token Prediction (AMTP) enhances language-model decoding by optimizing the token drafting process. It achieves significant speed improvements while maintaining accuracy across various settings.

Key facts

Read aloud

Adaptive Multi-Token Prediction AMTP enhances language-model decoding by optimizing the token drafting process. It achieves significant speed improvements while maintaining accuracy across various settings. Multi-token prediction MTP can accelerate language-model decoding by drafting several tokens and verifying them in a single target-model pass. We therefore formulate horizon selection as an online control problem, using verification feedback to learn how far the draft can be trusted in each context, and introduce Adaptive Multi-Token Prediction AMTP: a lightweight controller that requires no retraining and leaves the exact verifier unchanged. Across five draft-target pairings built on Llama-three point one-8B-Instruct with Medusa heads, Meta's MTP models, and Qwen3-4B-Inst-two thousand five hundred seven-MTP, evaluated on AIME twenty twenty five and GSM8K, AMTP decodes up to four.1x faster than standard decoding and matches or outperforms the fixed window in eight of ten settings, with throughput gains of up to ten percent.

Sources in this cluster