Running a 28.9M parameter LLM on an $8 microcontroller
8.4 relevance
Score Breakdown
technical depth 9
novelty 9
actionability 8
community 8
strategic 6
personal 9
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Running an LLM on an $8 microcontroller is highly novel, technically deep, and directly relevant to edge AI and open source.
Summary
An open-source project runs a 28.9M parameter LLM on an $8 ESP32-S3 microcontroller, achieving ~9.5 tokens/sec entirely on-device with no server. The breakthrough uses Google's Per-Layer Embeddings to store 25M parameters in flash, reading only ~450 bytes per token into SRAM, enabling a 100x parameter increase over prior microcontroller models. The model, trained on TinyStories, generates short coherent stories but cannot handle general Q&A or code.
Author
slvDev