AI TrackerFrontier desk · AEST
Back to news
normalaitech

Advancing SVD-based LLM Compression via Layer-Wise Error Model Search

arXiv cs.AI · 24 Oct 2026, 02:46 AEST

Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of LLMs. This work introduces Layer-wise Error Modeling Search (LEMS) and KFAC-SVD to address key limitations in existing methods.

Key facts

Read aloud

Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of L L M s. This work introduces Layer-wise Error Modeling Search LEMS and KFAC-SVD to address key limitations in existing methods. Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of L L M s. In this work, we address these limitations by introducing Layer-wise Error Modeling Search LEMS and KFAC-SVD. LEMS advances rank allocation by introducing a layer-wise error surrogate that integrates local and global layer importance alongside a propagation bias. Across Mistral, Qwen3, and Llama three model families, we show that LEMS consistently outperforms existing search strategies, delivering significant zero-shot accuracy gains of up to four point eight p.p. KFAC-SVD improves decomposition quality by utilizing token-wise statistics, mitigating the rank deficiency observed in prior Fisher-based SVD approaches.

Sources in this cluster