[GitHub Trending] lyogavin/airllm
8.1 relevance
Score Breakdown
technical depth 9
novelty 8
actionability 8
community 7
strategic 6
personal 9
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Enables LLM inference on limited hardware, highly relevant and innovative.
Summary
AirLLM is an open-source Python library that enables running massive LLMs—like 70B models on a 4GB GPU and even Kimi K3 (2.8T) on 3.72GB—by streaming layers or per-expert weights instead of loading the entire model, without quantization, distillation, or pruning. Its v3.0 adds FP8 support and one-line AutoModel inference for DeepSeek-V3 (671B on ~12GB) and Qwen3-235B (on ~3GB), while the latest update supports the largest open-source model, Kimi K3, via expert streaming on a single RTX 6000 Ada.
Author
lyogavin