AI TrackerFrontier desk · AEST
Back to news
normalaitech

Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers

arXiv cs.AI · 30 Oct 2026, 03:59 AEST

A new pixel-space generative Transformer model called SpecSlot has been introduced to enhance detail without increasing global attention costs. It achieves competitive performance on image datasets while reducing computational requirements.

Key facts

Read aloud

A new pixel-space generative Transformer model called SpecSlot has been introduced to enhance detail without increasing global attention costs. It achieves competitive performance on image datasets while reducing computational requirements. Pixel-space generative Transformers typically use one token per image patch, so capturing finer detail requires more tokens and makes global attention increasingly expensive. SpecSlot expands each spatial anchor into multiple slots, each assigned a disjoint band of DCT coefficients; this separates anchor density from spectral subdivision. Only the lowest-frequency slot from each anchor participates in global attention, while all slots interact locally. On ImageNet-two hundred fifty six, SpecSlot achieves a comparable final FID to a FLOPs-matched JiT control while converging faster. On text-to-image generation, SpecSlot surpasses JiT-B/eight on DPG-Bench by one point zero six at forty point six percent lower forward cost.

Sources in this cluster