Skip to content

One model gateway across clouds: LiteLLM on Cloud Run and Azure with zero API keys

8.1 relevance
Score Breakdown
technical depth
8
novelty
8
actionability
9
community
6
strategic
7
personal
10

Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.

Zero-API-key model gateway across clouds, directly actionable for cloud infrastructure and AI agents.

AI/ML dev.to
One model gateway across clouds: LiteLLM on Cloud Run and Azure with zero API keys
Summary

LiteLLM deployed on Cloud Run and Azure Container Apps serves as a model gateway between Vertex AI and Azure OpenAI, using a shared Cloud SQL Postgres database and Redis sidecars for state. Agents authenticate via Microsoft Entra tokens instead of API keys, with a custom auth hook that validates roles and enforces per-agent model permissions. During a demo, failing over one cloud's gateway was transparent to the calling agent.

Author

Miguel Moreno

More from Miguel Moreno →