A solo developer's inference engine is once again climbing GitHub Trending, and its premise is deliberately absurd: run a 744-billion-parameter frontier model on the laptop you already own, with no GPU and no Python at runtime.
GLM-5.2 is a mixture-of-experts (MoE) model. Each token activates only about 40B of its 744B parameters — roughly 5% — and only about 11GB of that changes from token to token. Colibri, written in pure C in a single file of roughly 1,300 lines, exploits exactly that sparsity. It keeps the static "dense" part — attention layers, shared experts and embeddings (~17B parameters, ~9.9GB at int4) — resident in RAM, while the 21,504 routed experts (~370GB at int4, about 19MB each, spread across 75 MoE layers of 256 experts plus an MTP head) stay on disk and are streamed on demand, with a per-layer LRU cache and the operating system's page cache acting as a free L2 tier.
The tradeoff is speed, and the author is honest about it: initial reports put throughput at roughly 0.05–0.1 tokens per second — usable as a demonstration, useless for interactive coding. Community benchmarks improve on NVMe drives with more RAM, but still sit far below cloud APIs. The README also carries an SSD-wear warning; heavy writes are limited to the KV cache, while expert reads dominate, and unified-memory machines such as Apple Silicon can keep the KV cache entirely in RAM.
The project is Apache-2.0 licensed, has about 28,600 stars, and drew 730+ points on Hacker News — where the top comment was simply "This is the hacker spirit." Work is underway to plug Colibri into coding agents such as opencode, and peers like antirez's ds4 explore the same disk-streaming idea. It makes little sense for production or cost optimization, but as a systems-level proof that MoE sparsity can put frontier-scale models on commodity hardware, it's a rare, readable artifact — an entire inference engine you can understand in an afternoon.
