The moment you build anything on top of more than one AI provider, you hit the same wall: every vendor has its own SDK, its own request shape, its own auth, its own quirks for streaming and tool calls. Switching from GPT to Claude to a local model means rewriting code, not changing a setting. LiteLLM erases that. It exposes 100+ models - OpenAI, Anthropic, Google, Mistral, Bedrock, Azure, and local backends like Ollama - behind a single OpenAI-style interface, as both a Python library and a standalone proxy server.
One Interface, Every Provider
In library form, the model is just a string. Change it and you've changed providers - the calling code stays identical:
from litellm import completion
# Today: a hosted model
resp = completion(model="gpt-4o", messages=[{"role": "user", "content": "Hello"}])
# Swap one string to route to a local Ollama model
resp = completion(model="ollama/llama3.2", messages=[{"role": "user", "content": "Hello"}])
The Proxy: A Gateway for a Whole Team
The proxy is where LiteLLM earns its keep in a team. You run it once, put your provider keys in it, and hand your developers a single endpoint and a virtual key. From there you get centralized spend tracking, per-key budgets and rate limits, fallbacks when a provider errors, and one place to see every request. Nobody has to hold the real vendor keys, and you can cap spend before a runaway loop does damage.
Fallbacks, Budgets, and Logging
Because everything flows through one layer, you can define an ordered fallback list (try one model, fall back to another, then to a local one), set hard monthly budgets per user or project, and stream logs to your observability stack of choice. It's the plumbing you'd otherwise write by hand for every provider.
Rough Edges and Limitations
- It normalizes, it doesn't equalize. Providers still differ in capability; a unified API can't make a weak model reason like a strong one.
- The proxy is one more service to run. For a solo script the library alone is plenty; the proxy pays off with a team or multiple apps.
- Edge features lag. Bleeding-edge, provider-specific features sometimes land in LiteLLM a little after the vendor ships them.
- Config grows. Fallbacks, budgets, and model aliases are powerful but the YAML can sprawl; keep it under version control.
If any part of your work touches more than one model - or you expect the "best" model to change every few months - LiteLLM is the abstraction that keeps that churn out of your application code.
Internal link suggestion: link "swap providers" to the Free Claude Code or Arena.ai post, which cover running the same workflow across different models.