One model gateway across clouds: LiteLLM on Cloud Run and Azure with zero API keys
8.1 relevance
Score Breakdown
technical depth 8
novelty 8
actionability 9
community 6
strategic 7
personal 10
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Zero-API-key model gateway across clouds, directly actionable for cloud infrastructure and AI agents.
Summary
LiteLLM deployed on Cloud Run and Azure Container Apps serves as a model gateway between Vertex AI and Azure OpenAI, using a shared Cloud SQL Postgres database and Redis sidecars for state. Agents authenticate via Microsoft Entra tokens instead of API keys, with a custom auth hook that validates roles and enforces per-agent model permissions. During a demo, failing over one cloud's gateway was transparent to the calling agent.