LiteLLM
LiteLLM is an open-source AI gateway providing unified, OpenAI-compatible API access to 100+ LLM providers, serving platform engineering and AI development teams with model routing, spend tracking, fallbacks, and enterprise governance.
- Company typePrivate
- Founded2023
- HeadquartersSan Francisco, United States
- Headcount1–10
- GTM typeB2B
- OfferingSoftware
What LiteLLM does
LiteLLM is an open-source AI gateway that provides a unified, OpenAI-compatible API interface across more than 100 large language model providers, including OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Mistral, and DeepSeek. Operated by Berrie AI Incorporated and founded in 2023, the company is headquartered in San Francisco, backed by Y Combinator as part of its W24 cohort, and operates with a small team of 1-10 employees. Its core product, the LiteLLM AI Gateway (Proxy), is an MIT-licensed Python/FastAPI server that handles model routing, automatic fallbacks during provider outages, per-key/user/team spend attribution, virtual key management with budgets and rate limits, LLM guardrails, Prometheus metrics, and audit logging. A complementary Python SDK enables direct provider-agnostic LLM calls without the proxy. The product has achieved substantial open-source distribution — 40K GitHub stars, 240M+ Docker pulls, 1B+ gateway requests, and 1,005+ contributors — making it one of the most widely adopted LLM routing layers in the developer ecosystem.
The company monetizes through a two-tier model: a free open-source core targeting self-serve developer adoption, and an Enterprise tier (custom pricing, annual contracts, 30-day trials) that adds SSO/SAML, audit logs, key rotation, custom SLAs, enhanced support, and is available cloud-hosted or self-hosted. Distribution combines PLG channels (GitHub, PyPI, Docker Hub) with enterprise direct sales and listings on AWS Marketplace and Azure Marketplace. The product portfolio has expanded beyond the core gateway into adjacent modules including a Rust-based performance-optimized gateway (early beta, ~15x throughput improvement), an Agent Platform for autonomous-agent lifecycle management (alpha), MCP Gateway, LiteLLM Skills, semantic caching on Valkey/ElastiCache, the LiteLLM Observatory load-testing system, and componentized data-plane/control-plane deployments. Strategic partnerships with CrowdStrike (security), Akto (AI agent security), and DigitalOcean (ecosystem) extend the platform's reach. Notable customers include Netflix and Lemonade. Compliance posture includes SOC 2 Type I/II and ISO 27001 certifications.
LiteLLM firmographics
Firmographics- Name
- LiteLLM
- Legal name
- Berrie AI Incorporated
- Website
- https://litellm.ai
- Company type
- Private
- Founded year
- 2023
- Operating status
- Operating
- Headcount range
- 1–10 employees
- Short description
- LiteLLM is an open-source AI gateway providing unified, OpenAI-compatible API access to 100+ LLM providers, serving platform engineering and AI development teams with model routing, spend tracking, fallbacks, and enterprise governance.
- Ownership category
- akta.pro rank
LiteLLM industry classification
Industry- Product category
- AI Infrastructure / LLM Gateway
- NAICS
- Computer Systems Design and Related Services (5415), Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services (518)
- SIC
- Services-Prepackaged Software (7372)
- akta.pro primary industry
- LLMOps & Generative AI Platforms (Prompt/Agent Orchestration, RAG) (HDAEANAD)
- akta.pro secondary industries
- MLOps/LLMOps & Model Lifecycle Management Services (BPAEAHAH), Network Performance, Visibility & SLA Monitoring (Managed NPM/Observability) (HDAIAFAI)
Keywords
Where LiteLLM is headquartered
LocationHeadquarters
- HQ city
- San Francisco
- HQ country
- United States
- HQ region
- North America
Markets served
LiteLLM business model
Business model- GTM type
- B2B
- Offering type
- Software
- Cost components
- Personnel, Technology or R&D, Infrastructure, Marketing or Sales, Operations
Revenue model
- Open Source (MIT License): Free, open-source AI gateway downloadable from GitHub, PyPI, and Docker Hub. Users self-host and manage their own infrastructure. Revenue is indirect through enterprise adoption leading to paid cloud/managed offerings.
- Enterprise Subscription: Enterprise tier with custom pricing including SSO/SAML, audit logs, custom SLAs, enhanced support, and all enterprise features. Sold via direct sales with 30-day trial. Available as cloud-hosted or self-hosted deployment. Revenue is subscription-based, billed annually or multi-year.
Pricing tiers
| Model | Billing | Price |
|---|---|---|
| Freemium | Others | Free, open-source AI gateway for developers |
| Subscription | Annual | Enterprise-grade AI gateway with custom SLAs and support |
Go-to-market motion2 records
Distribution channels4 records
Marketing channels6 records
LiteLLM product offering
Product offeringCore offering
LiteLLM provides an open-source AI gateway (Python FastAPI proxy server) and Python SDK that deliver a unified, OpenAI-compatible interface to call 100+ LLM providers (OpenAI, Anthropic, Azure, Bedrock, Gemini, Vertex, Mistral, DeepSeek, and others). The platform adds routing with automatic fallbacks, per-key/user/team spend tracking and attribution, virtual API key management with budgets and rate limits, LLM guardrails, Prometheus observability, audit logs, and enterprise controls (SSO/SAML, JWT, key rotation). A paid Enterprise tier adds custom SLAs, SSO, audit logs, and dedicated support, deployed cloud-hosted or self-hosted.
Product overview
LiteLLM is an AI gateway platform providing unified API access to 100+ LLM providers via OpenAI-compatible format. The product portfolio consists of the open-source MIT-licensed LiteLLM Python SDK (direct client library) and LiteLLM AI Gateway/Proxy (self-hosted gateway server) as core products, with LiteLLM Enterprise tier adding SSO/SAML, Audit Logs, Key Rotation, and custom SLAs. The LiteLLM Agent Platform (alpha) enables natural-language-driven AI agent management, built on the LiteLLM MCP Gateway and LiteLLM Skills for Anthropic Claude Code integration. LiteLLM Rust Gateway (early beta) offers a performance-optimized Rust implementation with sub-1ms overhead and 15x throughput improvement. Additional modules include LiteLLM Observatory (24-hour load testing), LiteLLM Semantic Caching (Valkey/AWS ElastiCache), LiteLLM Lite-Harness SDK (unified coding agent API), and LiteLLM Managed Agents Platform (alpha). The platform delivers AI gateway capabilities including spend tracking, rate limiting, load balancing, fallbacks, LLM guardrails, key management, and observability across OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, Azure, Mistral, DeepSeek, and 70+ other LLM providers.
Differentiator
Problem solved
Functional benefit
Products and services
- LiteLLM AI Gateway (Proxy) Unified AI gateway and proxy server providing OpenAI-compatible API access to 100+ LLM providers with model routing, automatic fallbacks, spend tracking, rate limiting, and observability. Designed for platform and DevOps teams and AI application developers who need centralized multi-provider LLM access. Offered as MIT-licensed open source and as an Enterprise tier.
- LiteLLM Python SDK Python client library for calling LLMs directly (without the proxy) via litellm.completion(), litellm.embeddings(), and other unified functions supporting 100+ providers with OpenAI-compatible format. Built for AI application developers who want unified calls inside their own code rather than a hosted proxy.
- LiteLLM Agent Platform Alpha-stage platform for deploying and managing autonomous AI agents with natural-language commands, enabling AI agents to create users, teams, keys, models, and MCP servers via the LiteLLM proxy. Targets organizations running agentic AI workflows.
- LiteLLM Rust Gateway Performance-optimized Rust implementation of the LiteLLM AI gateway offering sub-1ms overhead, sub-100MB memory footprint, 15x throughput improvement (453 to 6,782 requests/sec) and 11x memory reduction (359MB to 32MB). Maintains the same config.yaml, database, client API, and provider coverage as the Python gateway, targeting high-volume production LLM proxy deployments.
- LiteLLM Observatory Long-running 24-hour load-testing system used internally by LiteLLM to validate releases and catch regressions before deployment; positioned as part of the stability-first release process for the AI gateway.
- LiteLLM Semantic Caching Semantic caching layer on Valkey and AWS ElastiCache (Redis) that caches semantically similar LLM responses to reduce cost and latency by avoiding redundant provider calls across the gateway.
- LiteLLM MCP Gateway Model Context Protocol (MCP) gateway endpoint (/mcp) that enables AI agents to connect to tools and resources via the MCP standard, supporting Anthropic Skills API endpoints (/skills) and agent-to-agent (A2A) communication. Targets organizations running agentic AI workloads on LiteLLM.
Quantifiable outcome
- Day 0 model deployment — new LLM models available within 1 day of release without per-model integration work
- +4 more outcomes
Companies that use LiteLLM
Customer profileNamed customers2 records
Segments2 records
Ideal customer profiles2 records
LiteLLM technology and API
TechnologyTechnology focussed Yes
API detail
- Has API
- Yes
- API docs
- API detail
Core technology
AI maturity
App detail
Integration27 records
AI capability15 records
Feature9 records
LiteLLM partnerships and signals
Strategic signalPartnerships
One partnership is on record.
- CrowdStrikecoreCrowdStrike extended its Falcon AI Detection and Response product across LiteLLM's AI gateway to enable real-time threat detection, policy enforcement, and correlated telemetry for AI interactions. The integration places security controls at the AI gateway layer, inspecting AI traffic for prompt injection, data leakage, and manipulation risks, feeding data into CrowdStrike's Falcon Next-Gen SIEM product.
Scale indicators7 records
Recent moves6 records
Expansion highlights6 records
LiteLLM competitors and assessment
Company assessmentBroad incumbents
- Cloudflare AI Gateway: Cloudflare's AI Gateway provides caching, rate limiting, and analytics across LLM providers as part of Cloudflare's broader edge and developer platform. It overlaps directly with LiteLLM's gateway features but is bundled into a much larger platform.
- Google Vertex AI: Google Vertex AI provides model serving, routing across multiple foundation models, and enterprise governance features as part of Google Cloud. It competes as a broad incumbent platform offering capabilities overlapping with LiteLLM's gateway and enterprise tier.
- Kong (Kong AI Gateway): Kong is an established API gateway vendor that has extended its platform to provide AI/LLM routing, policy enforcement, and observability. It competes as a broader API infrastructure incumbent rather than an LLM-native specialist.
- AWS Bedrock: AWS Bedrock is a managed foundation-model service that includes model routing, guardrails, and cost controls as part of the AWS ecosystem. It serves as a broad incumbent alternative to LiteLLM for AWS-centric enterprises.
Emerging players
- Unify AI: Unify is an emerging AI routing and cost-optimization platform that routes requests across LLM providers for cost and latency optimization. It competes with LiteLLM's auto-routing, cost attribution, and budget enforcement features.
- Helicone: Helicone is an LLM observability and gateway platform offering request logging, caching, rate limits, and routing across providers. It overlaps with LiteLLM's observability, caching, and routing modules and targets a similar developer persona.
- TrueFoundry: TrueFoundry is an AI/ML deployment and serving platform that includes LLM gateway, routing, and governance features. It overlaps with LiteLLM's enterprise gateway, spend tracking, and governance modules.
- Martian: Martian provides intelligent LLM routing that automatically selects providers and models based on cost, latency, and quality. It overlaps with LiteLLM's model-routing and fallback capabilities for production AI workloads.
Direct peers
- OpenRouter: OpenRouter provides unified routing and billing across 100+ LLMs through a single OpenAI-compatible API. It competes head-on with LiteLLM's core value proposition of provider abstraction, with a hosted (vs. self-host) model.
- Portkey: Portkey is a venture-backed AI gateway offering unified LLM routing, observability, guardrails, and cost tracking across providers. It is the most direct functional competitor to LiteLLM's gateway/proxy product for production AI workloads.
Market position
Strengths5 records
Weaknesses5 records
Competitive moat5 records
Key risks7 records
Key highlights7 records
Customer concentration
LiteLLM social profiles
Digital presenceLiteLLM compliance and trust
Trust signalCompliance4 records
LiteLLM financial estimates
Financial estimateRevenue estimate
Valuation estimate
LiteLLM leadership team
Management profileNumber of profiles
Profiles2 records
LiteLLM funding detail
Funding detailFunding overview
Funding rounds
Investors
Funding detail is available on the Subscription and Enterprise plan.Contact sales →
LiteLLM M&A and investment
M&A and investmentM&A
Investments
M&A and investment is available on the Subscription and Enterprise plan.Contact sales →
Frequently asked questions about LiteLLM
What does LiteLLM do?
LiteLLM provides an open-source AI gateway (Python FastAPI proxy server) and Python SDK that deliver a unified, OpenAI-compatible interface to call 100+ LLM providers (OpenAI, Anthropic, Azure, Bedrock, Gemini, Vertex, Mistral, DeepSeek, and others). The platform adds routing with automatic fallbacks, per-key/user/team spend tracking and attribution, virtual API key management with budgets and rate limits, LLM guardrails, Prometheus observability, audit logs, and enterprise controls (SSO/SAML, JWT, key rotation). A paid Enterprise tier adds custom SLAs, SSO, audit logs, and dedicated support, deployed cloud-hosted or self-hosted.
Is LiteLLM a public or private company?
LiteLLM is a private company. It is classified as venture growth investor backed and is currently operating.
When was LiteLLM founded?
LiteLLM was founded in 2023. It employs 1 to 10 people.
Where is LiteLLM based?
LiteLLM is headquartered in San Francisco, United States, in the North America region.
How does LiteLLM make money?
Two revenue lines are on record. Open Source (MIT License) is the primary driver. The others are enterprise Subscription.
Who are LiteLLM's main competitors?
Broad incumbents on record are Cloudflare AI Gateway, Google Vertex AI, Kong (Kong AI Gateway) and AWS Bedrock. Emerging players are Unify AI, Helicone, TrueFoundry and Martian. Direct peers are OpenRouter and Portkey.
Does LiteLLM have an API?
Yes. REST API with OpenAI-compatible endpoints (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/responses, /v1/messages, /assistants, /audio/transcriptions, /audio/speech, /batches, /embeddings, /images, /rerank, /moderations, /skills, /search, /mcp, /a2a, /memory, /realtime, /rag/ingest, /rag/query, /vector_stores, /files, /fine_tuning, /evals). Supports Virtual Keys (sk- prefixed), OIDC/JWT-based Auth, Role-based Access Controls (RBAC), Service Accounts, and CLI/SSO authentication. Management API for keys, teams, users, models, spend logs, and audit logs. Separate Prometheus-compatible /metrics endpoint. Anthropic Skills API endpoints via /skills. MCP Gateway via /mcp. Developer documentation is at docs.litellm.ai/docs.
What industry is LiteLLM in?
LiteLLM's product category is AI Infrastructure / LLM Gateway. Its primary akta.pro industry code is HDAEANAD, LLMOps & Generative AI Platforms (Prompt/Agent Orchestration, RAG), with a secondary code of BPAEAHAH, MLOps/LLMOps & Model Lifecycle Management Services. Its NAICS code is 5415 and its SIC code is 7372.