Search
English

Open-source 'Strata' for running 125 billion parameter AI models locally on gaming PCs(github.com)

startupxo Discuss

Writing language: KoreanRead in the original language

Plain summary

- Strata is a free open-source project that enables large AI models with 125 billion parameters (Qwen3.8-Flash-Next) to run on typical gaming PCs running Windows or Linux with NVIDIA or AMD graphics cards of at least 12GB. - It requires at least 32GB RAM and about 80GB of storage space. During installation, it automatically detects the graphics card and recommends an appropriate model size and settings, supporting download and execution. - This model overcomes memory limits by utilizing GPU, RAM, CPU, and SSD in a distributed manner, reads long texts at over 1,000 tokens per second, and generates answers at speeds ranging from 53 to 93 tokens per second depending on GPU performance (benchmarks measured by the development team on two gaming PCs). - Compatible with OpenAI APIs for integration with other apps and coding agents, and since it runs locally, data is not transmitted to external clouds.

Read source ↗

Found on

Hacker News ↗ / 216 votes / 100 comments

GitHub ↗ / 16,916 stars

Have you run into something like this? Tell us in a line or two what you did. You do not need to know the product.

Sign in to comment

Keyboard shortcuts

Choose a post with J and K or the arrow keys, then press Enter.

K ↑ / J ↓
Previous post / next post
Enter C
Open summary and comments for the selected post
O
Open the original link of the selected post
/
Search
?
Show this help
Tab
Move to the submit or comment button, then press Enter

Type normally in text fields. Tab and Enter are always available.