SpecCTMC: Exact Speculative Simulation of Diffusion Language Model Continuous-Time Markov Chains
arXiv cs.CL · 6 Nov 2026, 03:41 AEST
SpecCTMC is a new speculative sampler designed to enhance the efficiency of masked diffusion language models. It reduces the need for sequential model evaluations while maintaining the integrity of the output distribution.
Key facts
- Masked diffusion language models offer parallel text generation, but independently sampling multiple tokens can disrupt their dependencies and change the output distribution.
- Exact continuous-time sampling avoids this approximation, yet typically requires one sequential model evaluation per token.
- SpecCTMC proposes successive token updates using previously computed model predictions, evaluates their intermediate states in parallel, and applies an exact acceptance-and-correction procedure.
- Experiments on LLaDA-8B-Instruct and Dream-v0-Instruct-7B across five benchmarks demonstrate 1.73-4.38x fewer sequential model evaluations than direct continuous-time sampling, while preserving the reference sampling distribution.
Read aloud
SpecCTMC is a new speculative sampler designed to enhance the efficiency of masked diffusion language models. It reduces the need for sequential model evaluations while maintaining the integrity of the output distribution. Masked diffusion language models offer parallel text generation, but independently sampling multiple tokens can disrupt their dependencies and change the output distribution. Exact continuous-time sampling avoids this approximation, yet typically requires one sequential model evaluation per token. SpecCTMC proposes successive token updates using previously computed model predictions, evaluates their intermediate states in parallel, and applies an exact acceptance-and-correction procedure. Experiments on LLaDA-8B-Instruct and Dream-v0-Instruct-7B across five benchmarks demonstrate one point seven three-four.38x fewer sequential model evaluations than direct continuous-time sampling, while preserving the reference sampling distribution.
Sources in this cluster
- SpecCTMC: Exact Speculative Simulation of Diffusion Language Model Continuous-Time Markov Chains
arXiv cs.CL · 6 Nov 2026, 03:41 AEST