AI TrackerFrontier desk · AEST
Back to news
normalaitech

Looped Uniform Diffusion Language Models

arXiv cs.CL · 22 Oct 2026, 03:24 AEST

Looped Uniform Diffusion Language Models (UDLMs) enhance training efficiency by applying selective looping during the denoising process. This method allows looped UDLMs to achieve significant performance improvements while using fewer training resources.

Key facts

Read aloud

Looped Uniform Diffusion Language Models UDLMs enhance training efficiency by applying selective looping during the denoising process. This method allows looped UDLMs to achieve significant performance improvements while using fewer training resources. Uniform diffusion language models UDLMs generate text by iteratively denoising sequences of ordinary vocabulary tokens, any of which may be corrupted and subsequently revised. We find that looping significantly improves training efficiency: looped UDLMs reach the validation NLL of a non-looped UDLM trained on about zero point five trillion tokens with up to two.8x fewer training FLOPs on OpenWebText and three.3x fewer on FineWeb-Edu. At a matched NFE budget, looped UDLMs reach up to nine point eight percent lower generation perplexity than the non-looped UDLM, with most gains achieved at moderate loop counts. On downstream tasks, this strategy uses twelve percent fewer loops than uniform loop-count sampling without degrading accuracy.

Sources in this cluster

Looped Uniform Diffusion Language Models · AI Tracker