Learning When to Commit: Schedule Distillation for Fast Diffusion Language Models
arXiv cs.AI · 27 Oct 2026, 03:35 AEST
A new framework called dSpark has been proposed for faster diffusion language generation. It focuses on learning when to commit tokens to improve prediction efficiency during decoding.
Key facts
- Diffusion large language models (dLLMs) can generate multiple tokens in parallel, but often require many decoding rounds to complete a response.
- Efficient generation depends not only on which tokens are easy to predict, but also on how committing to them helps predict the remaining positions.
- We propose dSpark, a new schedule distillation-based framework for learning when to commit tokens for faster diffusion language generation.
- On GSM8K, our method achieves remarkable 10.12× and 5.12× speedups over the various baselines on LLaDA-8B-Instruct and Dream-7B-Instruct, respectively, while maintaining superior task performance.
Read aloud
A new framework called dSpark has been proposed for faster diffusion language generation. It focuses on learning when to commit tokens to improve prediction efficiency during decoding. Diffusion large language models dLLMs can generate multiple tokens in parallel, but often require many decoding rounds to complete a response. Efficient generation depends not only on which tokens are easy to predict, but also on how committing to them helps predict the remaining positions. We propose dSpark, a new schedule distillation-based framework for learning when to commit tokens for faster diffusion language generation. On GSM8K, our method achieves remarkable ten point one two× and five point one two× speedups over the various baselines on LLaDA-8B-Instruct and Dream-7B-Instruct, respectively, while maintaining superior task performance.
Sources in this cluster
- Learning When to Commit: Schedule Distillation for Fast Diffusion Language Models
arXiv cs.AI · 27 Oct 2026, 03:35 AEST