Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
On-device model with confidence calibration, directly relevant to AI/ML and edge deployment.
Cactus Hybrid post-trains models with probes inside checkpoints to output confidence scores (0-1) as structured data, enabling on-device inference with automatic handoff to a larger model when confidence < 0.85. Using Gemma 4 E2B Hybrid, it routes only 15-35% of queries to Gemini 3.1 Flash-Lite to match its benchmarks across ChartQA, MMLU-Pro, and others, with handoff rates varying by quantization (FP16, 4-bit, 3-bit). The system supports Cactus, MLX, Transformers (with device_map caution), and llama.cpp via patched engine, and is available on Hugging Face.
cactus-compute