Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Hacker News · 8 Nov 2026, 20:22 AEST
Llama-modchip enables running 284B DeepSeek on a 64GB CPU without the need for a GPU. It supports various models and offers impressive performance metrics.
Key facts
- Run 284B DeepSeek-V4-Flash (<59GiB RAM, 44.4 tok/s pp, 9.1-14.7 tok/s) & 30B MoEs (25-111 tok/s) on a pure 64GB CPU at 256K ctx (0 GPUs).
- Llama-modchip is a set of ggml / llama.cpp kernel patches, quantization transcoders, and runtime tools designed to run sparse Mixture-of-Experts (MoE) checkpoints entirely in system RAM on x86_64 (AVX-512 / AVX2) and ARM64 (NEON / SVE / Apple Metal) processors.
- To answer the most common technical question upfront— exact bit-widths (bpw), base checkpoints, and what is compressed in each GGUF.
Read aloud
Llama-modchip enables running 284B DeepSeek on a 64GB CPU without the need for a G P U. It supports various models and offers impressive performance metrics. Run 284B DeepSeek-V4-Flash <59GiB RAM, forty four point four tok/s pp, nine point one-fourteen point seven tok/s & 30B MoEs twenty five-one hundred eleven tok/s on a pure 64GB CPU at 256K ctx zero G P U s. Llama-modchip is a set of ggml / llama.cpp kernel patches, quantization transcoders, and runtime tools designed to run sparse Mixture-of-Experts MoE checkpoints entirely in system RAM on x86 64 AVX-five hundred twelve / AVX2 and ARM64 NEON / SVE / Apple Metal processors. To answer the most common technical question upfront— exact bit-widths bpw, base checkpoints, and what is compressed in each GGUF.
Sources in this cluster
- Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Hacker News · 8 Nov 2026, 20:22 AEST