📄️ Router - Load Balancing
LiteLLM Router for load balancing, routing, retries, cooldowns, and fallbacks (failover) across multiple LLM deployments and providers.
📄️ [BETA] Adaptive Router
Beta feature. Share feedback on Discord or Slack.
📄️ [BETA] Request Prioritization
Beta feature. Use for testing only.
📄️ Auto Routing
One router for complexity, semantic, and adaptive routing. Classify each request with heuristics, an LLM classifier, or lexical/semantic keyword rules, then route to a pinned model, a random pool, or a Thompson-sampled pool per tier.
📄️ Semantic Auto Router (deprecated)
The semantic Auto Router is superseded by Auto Routing, which folds semantic keyword matching, complexity scoring, and adaptive routing into a single autorouter/complexity_router. New deployments should start there; the semantic router page is preserved for existing configs.
📄️ Proxy - Load Balancing
Load balance multiple instances of the same model
📄️ UI - Router Settings for Keys and Teams
Configure router settings at the key and team level to achieve granular control over routing behavior, fallbacks, retries, and other router configurations. This enables you to customize routing behavior for specific keys or teams without affecting global settings.
📄️ Budget Routing
LiteLLM Supports setting the following budgets:
📄️ Fallbacks (Provider Failover)
Set up automatic provider failover in LiteLLM. If a model or provider fails after num_retries, fallback to another model group for high availability and reliability.
📄️ [New] Fallback Management Endpoints
Dedicated endpoints for managing model fallbacks separately from the general configuration.
📄️ Tag Based Routing
Quick Start
📄️ Timeouts
The timeout set in router is for the entire length of the call, and is passed down to the completion() call level as well.
📄️ Provider specific Wildcard routing
Proxy all models from a provider
📄️ Health Check Driven Routing
Route traffic away from unhealthy deployments before users hit errors. Background health checks run on a configurable interval, and any deployment that fails gets removed from the routing pool proactively, not after a user request already failed.