CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs
arXiv cs.CL · 7 Nov 2026, 04:48 AEST
CodeQuant2 is a new framework designed to enhance outlier mitigation and adaptive clustering for low-precision large reasoning models. It aims to improve accuracy and efficiency in reasoning tasks while addressing the challenges of quantization.
Key facts
- Large reasoning models (LRMs) improve task performance by generating longer intermediate reasoning traces, but this substantially increases inference costs and makes low-precision quantization increasingly important.
- We propose CodeQuant2, a clustering-based LUT quantization framework for low-precision large reasoning models.
- CodeQuant2 mitigates activation outliers with rotation and scaling, uses clustering to represent non-uniform weight distributions, and adapts codebook sizes to the quantization difficulty of each layer.
- Combined with our customized LUT CUDA kernel, CodeQuant2 delivers substantial accuracy improvements on reasoning benchmarks while achieving up to $2.85\times$ speedup over BF16.
Read aloud
CodeQuant2 is a new framework designed to enhance outlier mitigation and adaptive clustering for low-precision large reasoning models. It aims to improve accuracy and efficiency in reasoning tasks while addressing the challenges of quantization. Large reasoning models LRMs improve task performance by generating longer intermediate reasoning traces, but this substantially increases inference costs and makes low-precision quantization increasingly important. We propose CodeQuant2, a clustering-based LUT quantization framework for low-precision large reasoning models. CodeQuant2 mitigates activation outliers with rotation and scaling, uses clustering to represent non-uniform weight distributions, and adapts codebook sizes to the quantization difficulty of each layer. Combined with our customized LUT CUDA kernel, CodeQuant2 delivers substantial accuracy improvements on reasoning benchmarks while achieving up to two dollars and eighty five cents\times$ speedup over BF16.
Sources in this cluster
- CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs
arXiv cs.CL · 7 Nov 2026, 04:48 AEST