LiteLLM is the popular open-source AI gateway — more than 61,000 stars on GitHub — that lets you call 100+ LLM providers through one OpenAI-format API, with cost tracking, guardrails and load balancing built in.
Every model provider ships a different API, pricing model and failure mode. LiteLLM normalizes all of it: a Python SDK and a high-performance proxy server (with a Rust core) expose Bedrock, Azure, OpenAI, Anthropic, Vertex AI, vLLM, NVIDIA NIM and many more behind a single OpenAI-compatible interface. Around that core it layers the things production teams need: per-key and per-team budgets, spend logging, caching, retries and fallbacks across providers, load balancing, guardrails and an MCP gateway. It has become default infrastructure in LLM stacks, trending 74 times in the archive, and is developed at a rapid pace with pushes in October 2026.
LiteLLM’s popularity comes from removing a whole category of glue code: instead of maintaining five provider SDKs and five billing dashboards, teams integrate once and get uniform errors, token counting and spend attribution. Its proxy also supports virtual keys with per-key budgets, which is how many organizations safely hand LLM access to many internal teams and products.
LiteLLM’s core is open source and free to self-host. LiteLLM Enterprise adds features such as SSO, audit logs and dedicated support, and a hosted proxy is available — see litellm.ai for enterprise pricing.
Platform teams and developers running multiple LLM providers in production who want one stable API plus the cost, reliability and governance controls a gateway provides.