Skip to content

Four Signals

Agentic insights for modern tech teams

Kimi K3, and what we can still learn from the pelican benchmark
General / simonwillison.net

Kimi K3, and what we can still learn from the pelican benchmark

Moonshot AI's Kimi K3 (2.8T parameters, open weights by July 27) surpasses DeepSeek's 1.6T v4 Pro and beats Claude Opus 4.8 and GPT-5.5 on self-reported benchmarks, but trails Claude Fable 5 and GPT-5.6 Sol. Priced at $3/$15 per million tokens (double Kimi K2.6), it leads Arena.ai's Frontend Code arena. The author's 'pelican' SVG benchmark, once correlated with model quality, now shows weaker correlation and fails to test agentic tool calling—yet running it via OpenRouter and LLM CLI reveals model characteristics like reasoning token costs (25 cents for 16,658 output tokens).

Why it matters

For a solutions architect focused on AI/ML agent orchestration and cloud infrastructure, this signals the rapid commoditization of large open-weight models and the need to evaluate models beyond simple benchmarks, focusing on agentic capabilities and cost-performance tradeoffs.
Prompt Injection Attacks Are Thwarting AI Hacking Agents
AI/ML / wired.com

Prompt Injection Attacks Are Thwarting AI Hacking Agents

Tracebit's 'context bombing' plants forbidden prompts alongside AWS secrets to trigger refusal mechanisms in LLM hacking agents, cutting admin privilege escalation from 57% to 5% across five models including Opus 4.8 and Gemini 3.1 Pro. The technique, building on their canary detection approach, reduced complete compromise (persistent foothold) from 36% to 1% in 152 simulated attack runs. This defensive prompt injection exploits the same vulnerability attackers use, turning AI agents' guardrails against them.
Porting a 128-expert MoE (Gemma-4 26B-A4B) to AWS Inferentia2 — where every rank weighted the wrong experts
Cloud / dev.to

Porting a 128-expert MoE (Gemma-4 26B-A4B) to AWS Inferentia2 — where every rank weighted the wrong experts

Porting Gemma-4 26B-A4B's 128-expert MoE to AWS Inferentia2 required a workaround for its sparse top-8 routing: computing all 128 experts per token and masking non-selected ones with zero weights, collapsing the expert loop into two standard parallel linear layers (ColumnParallel and RowParallel) that Neuron's ModelBuilder can shard. The architecture also includes a shared dense MLP in parallel with the MoE per layer, and despite passing unit tests with MAXDIFF ≈ 2e-6, the first device run yielded empty output due to a bug in an unexpected location. After fixing, the model achieved SEQ_MATCH on greedy decode with 77 ms prefill.
General / curiouscoding.nl

Static search trees: 40x faster than binary search (2024)

An S+ tree implementation in Rust achieves 40x throughput improvement over binary search by combining a B-tree-like layout (node size B=15), SIMD vectorization, batching, and cache-conscious prefetching. The design optimizes for high-throughput sorted search, using popcount-based branching and manual SIMD to minimize instructions per query. The code is open-source and targets applications like suffix array search in bioinformatics.
From intent to enforcement: Lessons from operating Kubernetes controllers at scale
AI/ML / thenewstack.io

From intent to enforcement: Lessons from operating Kubernetes controllers at scale

Kubernetes controllers at scale face correctness challenges from cache staleness and object churn. Amazon EKS's Network Policy Controller centralizes label-to-IP resolution for eBPF enforcement, while the VPC Resource Controller assigns individual AWS security groups to pods, avoiding node-level inheritance that would leak access. These patterns preserve intent as clusters grow to thousands of nodes.
Instrumenting an AI-Powered GitHub Analyzer with OpenTelemetry and SigNoz
AI/ML / dev.to

Instrumenting an AI-Powered GitHub Analyzer with OpenTelemetry and SigNoz

GitIntel, an AI-powered GitHub analyzer using Gemini 2.5 Flash to score repos across 8 engineering dimensions, suffered from opaque pipeline performance with token usage varying 4k-40k and latency 8-52 seconds. By instrumenting with OpenTelemetry and a self-hosted SigNoz instance, the developer gained end-to-end traces correlating GitHub API calls and LLM interactions, revealing which steps caused bottlenecks. This replaced log-based guesswork with precise visibility into execution flow, token consumption, and failure context.
Five Gemma-4 models, one accelerator: what porting E2B 31B to AWS Inferentia2 taught me
AI/ML / dev.to

Five Gemma-4 models, one accelerator: what porting E2B 31B to AWS Inferentia2 taught me

The author hand-ported all five Gemma-4 models (E2B, E4B, 12B, 31B, 26B-A4B MoE) to AWS Inferentia2 because vendor stacks (optimum-neuron, Neuron vLLM) lack support for architectures like cross-layer KV-sharing and 128-expert MoE. The ports share a skeleton of softcap 30, 262K vocabulary, GQA, and interleaved sliding+global attention, but differ in approach: E2B/E4B use torch_neuronx.trace with per-layer embeddings, 12B forces eager attention, 31B requires NxD ModelBuilder with TP=8, and the MoE model uses ModelBuilder with top-8 routing. All five decode token-for-token identically to CPU fp32 reference, with softcap dropped from the device graph.
Your page ranks #1 on Google and is invisible to ChatGPT. Both are true.
AI/ML / dev.to

Your page ranks #1 on Google and is invisible to ChatGPT. Both are true.

AI crawlers (GPTBot, ClaudeBot) never execute JavaScript, confirmed by a Vercel/MERJ study showing 11.5% of GPTBot fetches download JS but don't run. This means client-side rendered React sites ranking #1 on Google are invisible to ChatGPT, especially since 92% of ChatGPT queries rely on Bing's limited JS rendering. The fix remains server-side rendering or prerendering to embed content in the initial HTML.
“They’re dead if they don’t offer this”: DoorDash’s CLI for agents may be out of necessity
AI/ML / thenewstack.io

“They’re dead if they don’t offer this”: DoorDash’s CLI for agents may be out of necessity

DoorDash released dd-cli, a command-line interface enabling AI agents to autonomously place orders—searching stores, comparing prices, and checking out without human approval. Co-founder and CTO Andy Fang announced the limited beta for macOS developers in the US and Canada via waitlist. The move sparks debate: Gergely Orosz warns it risks disintermediating DoorDash's low-margin, ad-driven customer relationship, while Dax Raad counters that refusing to offer a sanctioned agent interface would be fatal if agent-led ordering becomes the norm.
My Inferentia port matched its reference token-for-token — and still output garbage
AI/ML / dev.to

My Inferentia port matched its reference token-for-token — and still output garbage

Porting Google's Gemma-4 31B dense model (60 layers, ~60GB bf16) to AWS Inferentia2 (inf2.24xlarge, TP=8) produced token-for-token identical output to CPU but was gibberish. Manual parallel_model_trace failed due to OOM from 8 simultaneous fp32 compiles and deadlock on serialized ranks, forcing a pivot to ModelBuilder which compiles one rank and loads weights per rank. Attention layout issues emerged: 50 sliding layers (head_dim 256, 16 KV heads) and 10 global layers (head_dim 512, 4 KV heads) — the 4 KV heads cannot be sharded across 8 ranks, requiring replication.