Open source claims to run a 284 billion-parameter DeepSeek model on a PC with 64GB memory without a GPU(github.com)
Writing language: KoreanRead in the original language
Plain summary
- llama-modchip, released by developer synchopate, is open source software (Apache-2.0) that compresses AI models further and adjusts CPU computation to let people run large AI models locally without a graphics card.
- According to the developer, DeepSeek V4 Flash shrinks from 76.87GiB to 56.56GiB, and peak memory use is about 57.6GiB even with a 256K context, so it fits on a 64GB PC. The original text says the existing 76.87GiB version stops due to insufficient memory on 64GB Linux.
- README figures show input processing at about 44tok/s and response generation at 9.06~10.74tok/s on an AMD Ryzen 9 9955HX (16 cores, 64GB DDR5). With an auxiliary draft technique, it reaches 11.5~14.7tok/s. These are the developer's own measurements.
- Nemotron 3.5 30B at the 30 billion scale was reportedly reduced to 14.35GiB and achieved 25~60tok/s on a PC with 16~32GB memory. The compressed models use a proprietary format and cannot be opened in standard llama.cpp, Ollama or LM Studio.
Editorial opinion
O: 64GB For developers who want to try a large model on a desktop without a GPU, the suggestion is to run the CPU check script in the README, then start by measuring the speed of the 14.35GiB 30B model.
X: The compressed model uses the dedicated GGML_TYPE_Q2_0 format, so it cannot be opened in standard llama.cpp, Ollama, or LM Studio. It must be run using the provided dedicated binary or the modchip command.
Writing language: Korean
Found on
Hacker News ↗ / 237 votes / 72 comments
Have you run into something like this? Tell us in a line or two what you did. You do not need to know the product.