- Janas keeps model files on disk and reads from the SSD only the expert parts needed to generate each answer, so it can run mixture-of-experts models larger than memory on a laptop. A mixture-of-experts model uses only some of its many expert parts, and frequently used parts are kept in memory.
- The development team reported, from measurements on its own laptop (Intel Core Ultra 9 185H, 32GB memory, NVMe SSD), that the generation speed for the 80B-class Qwen3-Next model was 24 tok/s with the draft function turned off, while llama.cpp was 2.8 tok/s. tok/s means tokens generated per second. The original says these are the team's measurements and results may differ on other devices.
- Executable files for Linux x86-64 (Ubuntu 22.04, Debian 12 or later), Windows 10 (1903 or later) and 11 are posted on GitHub Releases, and models are obtained from Hugging Face in converted form. The source code is published under the GPL-3.0 license.
O: For developers who want to run an 80B-class model on a laptop or a 32GB-memory PC without the cloud, the janas-bench in the original post is useful for first measuring the speed of their own machine.
X: The original post says it is still experimental and that a desktop was once pushed into swapping or froze after a GPU driver reset, so before running it on a machine you depend on for important work, you should weigh this risk first.