AI TrackerFrontier desk · AEST
Back to news
normalaitech

ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models

arXiv cs.CL · 15 Oct 2026, 03:08 AEST

ReaDy introduces a new speculative decoding method that utilizes hybrid linear attention models for improved context retrieval during token drafting. This approach significantly accelerates autoregressive generation while maintaining efficiency in computation and I/O overhead.

Key facts

Read aloud

ReaDy introduces a new speculative decoding method that utilizes hybrid linear attention models for improved context retrieval during token drafting. This approach significantly accelerates autoregressive generation while maintaining efficiency in computation and I/O overhead. Speculative decoding accelerates autoregressive generation by drafting multiple tokens per step, yet feature-based drafters rely on target hidden features whose contextual emphasis may not align with the needs of subsequent draft positions. Hybrid linear attention HLA models maintain compact recurrent states that accumulate historical associations, offering an efficient source of complementary context for speculative drafting. ReaDy equips each draft position with a learned query to retrieve supplementary context directly from the target's committed recurrent states. Across seven benchmarks spanning mathematics, code generation, and dialogue, ReaDy achieves an average three dollars and sixteen cents imes$ decoding speedup over autoregressive decoding with Qwen3.eight-27B on a single NVIDIA B200 G P U.

Sources in this cluster