Rethinking the design space of Vanilla Pixel Diffusion Transformers
arXiv cs.AI · 17 Oct 2026, 03:59 AEST
Pixel-space diffusion models are gaining traction as an alternative to latent diffusion by eliminating the lossy latent bottleneck. The introduction of MFlow allows for adaptive parameterization that enhances generative performance across various datasets.
Key facts
- JiT shows that a vanilla diffusion transformer can generate directly in pixel space by replacing the commonly used velocity prediction ($\boldsymbol{u}$-prediction) with clean image prediction ($\boldsymbol{x_0}$-prediction).
- We find that the preferred parameterization varies systematically with the data representation and model capacity -- notably, increasing model capacity shifts preference toward velocity prediction.
- We evaluate MFlow across ImageNet, UCF-101 video, and large-scale text-to-image generation, demonstrating competitive or improved generative performance across diverse settings.
- In addition, we find that vanilla pixel diffusion transformers, irrespective of parameterization, retain a persistent high-frequency deficit relative to real images.
- With only a small amount of additional training, ConvRefiner restores the high-frequency statistics of generated images while preserving FID.
Read aloud
Pixel-space diffusion models are gaining traction as an alternative to latent diffusion by eliminating the lossy latent bottleneck. The introduction of MFlow allows for adaptive parameterization that enhances generative performance across various datasets. JiT shows that a vanilla diffusion transformer can generate directly in pixel space by replacing the commonly used velocity prediction $\boldsymbol u $-prediction with clean image prediction $\boldsymbol x 0 $-prediction. We find that the preferred parameterization varies systematically with the data representation and model capacity -- notably, increasing model capacity shifts preference toward velocity prediction. We evaluate MFlow across ImageNet, UCF-one hundred one video, and large-scale text-to-image generation, demonstrating competitive or improved generative performance across diverse settings. In addition, we find that vanilla pixel diffusion transformers, irrespective of parameterization, retain a persistent high-frequency deficit relative to real images. With only a small amount of additional training, ConvRefiner restores the high-frequency statistics of generated images while preserving FID.
Sources in this cluster
- Rethinking the design space of Vanilla Pixel Diffusion Transformers
arXiv cs.AI · 17 Oct 2026, 03:59 AEST