There Is No Free Loop: When Does Looping a Trained Model Pay for Its Inference Compute?
arXiv cs.AI · 31 Oct 2026, 02:56 AEST
Looping a trained model allows each token to pass through a block of layers multiple times without increasing the parameter count. However, this approach only proves beneficial for smaller models, as larger models do not outperform their unlooped counterparts.
Key facts
- Looping re-applies a block of layers so that each token passes through the block more than once.
- Adding a loop to a trained model is attractive because it leaves the parameter count, and hence the disk size, the memory footprint and the training cost unchanged.
- Across Qwen3 and Qwen3.5 models from 0.8B to 9B parameters, only the looped 0.8B model beats both baselines.
- At 4B and 9B, the looped model does not beat the first baseline, i.e. the same model trained without the loop.
- Introducing loops in an already pretrained model therefore only pays for small models with a fixed parameter budget.
Read aloud
Looping a trained model allows each token to pass through a block of layers multiple times without increasing the parameter count. However, this approach only proves beneficial for smaller models, as larger models do not outperform their unlooped counterparts. Looping re-applies a block of layers so that each token passes through the block more than once. Adding a loop to a trained model is attractive because it leaves the parameter count, and hence the disk size, the memory footprint and the training cost unchanged. Across Qwen3 and Qwen3.five models from zero.8B to 9B parameters, only the looped zero.8B model beats both baselines. At 4B and 9B, the looped model does not beat the first baseline, i.e. the same model trained without the loop. Introducing loops in an already pretrained model therefore only pays for small models with a fixed parameter budget.
Sources in this cluster
- There Is No Free Loop: When Does Looping a Trained Model Pay for Its Inference Compute?
arXiv cs.AI · 31 Oct 2026, 02:56 AEST