[GitHub Trending] jingyaogong/minimind

7.5 relevance

Train a 64M LLM from scratch in 2h, novel and relevant to AI trends.

AI/ML github.com

Answer the board on AI readiness

Arrive with evidence, not opinion. See how your foundation benchmarks against your peers, get a clear read, and leave with a ranked place to start.

Why It Matters

For platform and AI engineers, this provides a fully reproducible, low-cost sandbox to understand and experiment with every stage of LLM training—from data cleaning to agentic reinforcement learning—without relying on black-box abstractions, making it ideal for prototyping and teaching.

Key Takeaways

Clone the repo and run the full training pipeline on a single GPU to gain hands-on understanding of LLM internals from tokenization to RLHF.

🧠 Train a 64M-parameter LLM from scratch in just 2h! - jingyaogong/minimind

Summary

MiniMind is an open-source project that trains a 64M-parameter language model from scratch for under $3 in GPU rental and 2 hours on a single NVIDIA 3090. It covers the full LLM pipeline—pretrain, SFT, LoRA, RLHF (DPO), RLAIF (PPO/GRPO/CISPO), Tool Use, Agentic RL, and distillation—using pure PyTorch without high-level abstractions from libraries like transformers or trl. The latest release aligns with the Qwen3/Qwen3-MoE architecture and adds native Agentic RL training with multi-turn tool-use support.

Author

jingyaogong