ποΈ Router - Load Balancing
LiteLLM Router for load balancing, routing, retries, cooldowns, and fallbacks (failover) across multiple LLM deployments and providers.
ποΈ [BETA] Adaptive Router
Beta feature. Share feedback on Discord or Slack.
ποΈ [BETA] Request Prioritization
Beta feature. Use for testing only.
ποΈ Auto Routing
One router for complexity, semantic, and adaptive routing. Classify each request with heuristics, an LLM classifier, or lexical/semantic keyword rules, then route to a pinned model, a random pool, or a Thompson-sampled pool per tier.
ποΈ Semantic Auto Router (deprecated)
The semantic Auto Router is superseded by Auto Routing, which folds semantic keyword matching, complexity scoring, and adaptive routing into a single autorouter/complexity_router. New deployments should start there; the semantic router page is preserved for existing configs.
ποΈ Proxy - Load Balancing
Load balance multiple instances of the same model
ποΈ UI - Router Settings for Keys and Teams
Configure router settings at the key and team level to achieve granular control over routing behavior, fallbacks, retries, and other router configurations. This enables you to customize routing behavior for specific keys or teams without affecting global settings.
ποΈ Budget Routing
LiteLLM Supports setting the following budgets:
ποΈ Fallbacks (Provider Failover)
Set up automatic provider failover in LiteLLM. If a model or provider fails after num_retries, fallback to another model group for high availability and reliability.
ποΈ [New] Fallback Management Endpoints
Dedicated endpoints for managing model fallbacks separately from the general configuration.
ποΈ Tag Based Routing
Quick Start
ποΈ Timeouts
The timeout set in router is for the entire length of the call, and is passed down to the completion() call level as well.
ποΈ Provider specific Wildcard routing
Proxy all models from a provider
ποΈ Health Check Driven Routing
Route traffic away from unhealthy deployments before users hit errors. Background health checks run on a configurable interval, and any deployment that fails gets removed from the routing pool proactively, not after a user request already failed.