LiteLLM

LiteLLM is an open source AI Gateway and Python SDK built by BerriAI, backed by Y Combinator (W23), that gives platform and ML teams a single unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and many more — all using the OpenAI format. It is available as a self-hosted proxy server or as a Python library, and the GitHub repository has accumulated over 47,000 stars and more than 1,000 contributors as of mid-2026.

What It Is

LiteLLM sits between your application code and the fragmented landscape of LLM provider APIs. Instead of maintaining separate SDKs, authentication patterns, and request formats for every model provider, developers and platform teams route all LLM calls through LiteLLM's unified interface. The project ships in two complementary forms: a Python SDK for direct library integration in application code, and an AI Gateway (Proxy Server) that acts as a centralized service for entire teams or organizations.

Core Capabilities

The open source tier covers a broad set of production-ready features:

• 100+ LLM provider integrations — OpenAI, Anthropic, Azure, Bedrock, Vertex AI, Cohere, Mistral, Groq, Ollama, HuggingFace, and dozens more, across endpoints including /chat/completions, /embeddings, /images, /audio, /batches, /rerank, and the A2A agent protocol.
• Drop-in OpenAI compatibility — swap providers without rewriting application code; the proxy exposes an OpenAI-compatible API surface.
• Virtual keys — issue scoped API keys to teams and users without exposing underlying provider credentials.
• Spend tracking and budgets — attribute cost to individual keys, users, teams, or organizations; supports tag-based tracking and logging to S3, GCS, and other destinations.
• Load balancing and fallbacks — route across multiple deployments with configurable retry and fallback logic; the README cites 8ms P95 latency at 1,000 RPS in published benchmarks.
• LLM guardrails — set content and safety guardrails per request or per key/team.
• Observability — integrates with Langfuse, Langsmith, Arize Phoenix, OpenTelemetry, Datadog, Prometheus, and more.
• MCP Gateway — connect MCP servers to any LLM through the proxy, with Cursor IDE support documented.
• A2A Agent Gateway — invoke agents built on LangGraph, Vertex AI Agent Engine, Azure AI Foundry, Bedrock AgentCore, and Pydantic AI via the A2A protocol.

Deployment Model

LiteLLM is designed to be self-hosted. The proxy server ships as a Docker image (the homepage states 240M+ Docker pulls), and the README documents quick-start paths via Docker Compose, Render, and Railway. A -stable Docker tag undergoes 12-hour load tests before publication. An enterprise-hosted (SaaS) option is also available for organizations that prefer not to manage infrastructure themselves.

Enterprise Tier

The enterprise offering adds features on top of the open source base, available in both cloud and self-managed deployment:

• SSO and SCIM provisioning
• OIDC/JWT authentication with group-based access for automatic token generation
• Audit logs and secret manager integrations
• Virtual key rotation
• Team and org admin delegation
• Pagerduty alerting and advanced Prometheus metrics
• 24/7 support with custom SLAs and live upgrade assistance

The enterprise page lists a 30-day trial key available via a sales interest form, and pricing is available on request.

Update: v1.85.0

The latest release as of mid-May 2026 is v1.85.0, published on May 17, 2026. The repository is actively maintained on the litellm_internal_staging branch, with the last push recorded on May 20, 2026. The project's GitHub topics include mcp-gateway and a2a, reflecting recent additions of MCP and agent-to-agent protocol support as product-direction signals. The changelog is published at docs.litellm.ai/release_notes, and a town hall to discuss the roadmap was announced for May 19, 2026.