[GitHub Trending] JustVugg/colibri

7.7 relevance

Novel MoE inference engine in C with zero deps, directly relevant to running large models on edge hardware.

AI/ML github.com

Answer the board on AI readiness

Arrive with evidence, not opinion. See how your foundation benchmarks against your peers, get a clear read, and leave with a ranked place to start.

Why It Matters

For platform and infrastructure engineers, Colibrì demonstrates a path to running multi-hundred-billion-parameter models on commodity hardware, directly challenging the hyperscaler-gatekept inference model and enabling private, auditable frontier AI workloads without costly GPU clusters.

Key Takeaways

Evaluate Colibrì's multitiering (JIT weight prefetch, LRU routing heat, and CPU/GPU overlap) for your on-prem or edge inference pipeline to cut VRAM dependency while preserving model semantics.

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 - JustVugg/colibri

Summary

Colibrì is a pure-C inference engine that runs frontier MoE models (744B–2.8T parameters) on consumer hardware by treating storage, RAM, and VRAM as a single multitier memory hierarchy. It supports nine model families including GLM-5.3, Kimi K3, and DeepSeek V4.1 Flash, with a web dashboard showing per-expert routing heat, tier usage, and latency breakdown. The engine is an open research platform that deliberately trades speed for semantic fidelity—no silent precision drops or router changes—and measures end-to-end performance on real hardware rather than microbenchmarks.

Author

JustVugg