[GitHub Trending] MakazhanAlpamys/Soup

7.3 relevance

Fine-tune LLMs from YAML with layer streaming on low-end GPU is novel and actionable for AI engineers.

AI/ML github.com

Answer the board on AI readiness

Arrive with evidence, not opinion. See how your foundation benchmarks against your peers, get a clear read, and leave with a ranked place to start.

Why It Matters

For a Solutions Architect focused on AI/ML workflows and developer experience, Soup directly addresses the 30-50% infrastructure overhead teams face in LLM fine-tuning, enabling local experimentation without cloud GPU dependency and providing reliable evaluation gates that catch regressions and caller errors.

Key Takeaways

Evaluate Soup's layer streaming and evaluation gate for your fine-tuning pipeline to reduce infrastructure friction and ensure model improvements are statistically significant.

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. - MakazhanAlpamys/Soup

Summary

Soup is an open-source CLI tool that fine-tunes and post-trains LLMs with a single command, eliminating SSH and configuration overhead. It achieves 119.6 tok/s for Llama-3.1-8B-Instruct on a 4 GB laptop GPU via layer streaming, which keeps frozen base layers out of VRAM, and supports QLoRA, auto batch size, and quantization. The v0.73.2 release fixes evaluation gate bugs—including a suite that scored 0.225 for a 40/40 correct model due to brace hygiene parsing—adds a benign-prompt refusal axis to prevent gaming, and introduces `--noise-floor N` to reject deltas smaller than GPU decoding variance (0.015–0.020 spread).

Author

MakazhanAlpamys