Skip to content

Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

8.7 relevance
Score Breakdown
technical depth
9
novelty
9
actionability
8
community
9
strategic
7
personal
10

Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.

On-device model with confidence calibration, directly relevant to AI/ML and edge deployment.

AI/ML github.com
On-device models that know when they're wrong: every answer carries a confidence score for cloud handoff. - cactus-compute/cactus-hybrid
Summary

Cactus Hybrid post-trains models with probes inside checkpoints to output confidence scores (0-1) as structured data, enabling on-device inference with automatic handoff to a larger model when confidence < 0.85. Using Gemma 4 E2B Hybrid, it routes only 15-35% of queries to Gemini 3.1 Flash-Lite to match its benchmarks across ChartQA, MMLU-Pro, and others, with handoff rates varying by quantization (FP16, 4-bit, 3-bit). The system supports Cactus, MLX, Transformers (with device_map caution), and llama.cpp via patched engine, and is available on Hugging Face.

Author

cactus-compute