Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
8.4 relevance
Score Breakdown
technical depth 8
novelty 9
actionability 8
community 8
strategic 8
personal 10
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Needle: distilled tool-calling model, perfect alignment with AI agent orchestration and open source.
Summary
Needle is a 26M parameter open-source model distilled from Gemini for single-shot tool calling, using a Simple Attention Network with encoder-decoder, GQA+RoPE, ZCRMSNorm, and tied embeddings. Pretrained on 16 TPU v6e for 200B tokens then fine-tuned on 2B tokens of function calls, it achieves 6000 toks/sec prefill and 1200 decode on Cactus, outperforming FunctionGemma-270m and Qwen-0.6B on tool calling while being finetunable on a Mac/PC.