AI TrackerFrontier desk · AEST
Back to news
normalaitech

Low-Bit N-Gram Embeddings for Language Models

arXiv cs.AI · 23 Oct 2026, 03:56 AEST

Recent research explores low-bit n-gram embeddings to reduce memory costs in language models. The study finds that four-bit tables can save approximately 75% of memory while maintaining over 99% of performance improvements.

Key facts

Read aloud

Recent research explores low-bit n-gram embeddings to reduce memory costs in language models. The study finds that four-bit tables can save approximately seventy five percent of memory while maintaining over ninety nine percent of performance improvements. Many recent open-weight language models use n-gram embedding tables to improve performance. We propose storing n-gram embeddings at low precision and systematically investigate different quantization methods and bit allocations across tables. Across nanochat-, Llama-, and GPT-two-family models trained from scratch, four-bit tables save approximately seventy five percent of table memory while preserving more than ninety nine percent of the validation-loss improvement from adding the tables. These findings establish low-bit precision as a practical way to reduce the memory requirements of n-gram embeddings and use the saved capacity to improve model performance.

Sources in this cluster