跳至主要內容

/rag/query

RAG 查詢端點：搜尋向量儲存 →（重新排序）→ LLM 完成

功能支援
記錄是
串流是
重新排序是（可選）
支援的提供者openai, bedrock, vertex_ai

快速開始​

RAG Query with OpenAI
curl -X POST "http://localhost:4000/v1/rag/query" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "gpt-5.6-luna",
        "messages": [{"role": "user", "content": "What is LiteLLM?"}],
        "retrieval_config": {
            "vector_store_id": "vs_abc123",
            "custom_llm_provider": "openai",
            "top_k": 5
        }
    }'

運作方式​

RAG 查詢端點會執行以下步驟：

  1. 擷取查詢：從最後一則使用者訊息擷取查詢文字
  2. 搜尋向量儲存：在指定的向量儲存中搜尋相關內容
  3. 重新排序（可選）：使用重新排序模型對搜尋結果重新排序
  4. 產生回應：以擷取到的內容前置於訊息之前呼叫 LLM

回應​

回應遵循標準 OpenAI 聊天完成格式，並包含額外的搜尋中繼資料：

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1703123456,
  "model": "gpt-5.6-luna",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "LiteLLM is a unified interface for 100+ LLMs..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 150,
    "completion_tokens": 50,
    "total_tokens": 200
  },
  "_hidden_params": {
    "search_results": {...},
    "rerank_results": {...}
  }
}

搭配重新排序​

加入 rerank 設定以提升結果品質：

RAG Query with Reranking
curl -X POST "http://localhost:4000/v1/rag/query" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "gpt-5.6-luna",
        "messages": [{"role": "user", "content": "What is LiteLLM?"}],
        "retrieval_config": {
            "vector_store_id": "vs_abc123",
            "custom_llm_provider": "openai",
            "top_k": 10
        },
        "rerank": {
            "enabled": true,
            "model": "cohere/rerank-english-v3.0",
            "top_n": 3
        }
    }'

串流​

啟用串流以取得即時回應：

RAG Query with Streaming
curl -X POST "http://localhost:4000/v1/rag/query" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "gpt-5.6-luna",
        "messages": [{"role": "user", "content": "What is LiteLLM?"}],
        "retrieval_config": {
            "vector_store_id": "vs_abc123",
            "custom_llm_provider": "openai"
        },
        "stream": true
    }'

篩選檢索​

設定 retrieval_config.retrieval_filter 以將搜尋範圍縮小到其中繼資料相符的區塊。此篩選條件會以 filters 引數傳遞給向量儲存，因此它會採用提供者接受的任何形式：Bedrock Knowledge Bases 接受其原生運算子（equals、notEquals、andAll、orAll 等；andAll 或 orAll 至少需要兩個子句）以及 OpenAI 比較格式（{"key": ..., "value": ..., "operator": "eq"}），而 OpenAI 向量儲存則接受其自己的篩選物件。retrieval_config.filters 可作為別名接受；當兩個鍵都設定時，retrieval_filter 會生效。若向量儲存拒絕該篩選條件，請求會以該儲存的狀態碼和訊息失敗

RAG Query with a Bedrock metadata filter
curl -X POST "http://localhost:4000/v1/rag/query" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "bedrock/us.anthropic.claude-sonnet-5",
        "messages": [{"role": "user", "content": "How do I reset my password?"}],
        "retrieval_config": {
            "vector_store_id": "KNOWLEDGE_BASE_ID",
            "custom_llm_provider": "bedrock",
            "top_k": 5,
            "retrieval_filter": {
                "andAll": [
                    {"equals": {"key": "department", "value": "billing"}},
                    {"equals": {"key": "doc_type", "value": "manual"}}
                ]
            }
        }
    }'

請求參數​

頂層​

參數型別必填說明
modelstringYes用於生成的 LLM 模型
messagesarrayYes聊天訊息陣列（OpenAI 格式）
retrieval_configobjectYes向量儲存搜尋設定
rerankobjectNo重新排序設定
streambooleanNo啟用串流（預設：false）

retrieval_config​

參數類型預設值說明
vector_store_idstringrequired要搜尋的向量儲存 ID
custom_llm_providerstring"openai"向量儲存提供者
top_kinteger10要擷取的結果數量
retrieval_filterobject-轉送至向量儲存搜尋的中繼資料篩選條件（請參閱［篩選檢索］(#filtering-retrieval)）
filtersobject-retrieval_filter 的別名；當兩者都設定時，retrieval_filter 會生效

rerank​

參數型別預設說明
enabledbooleanfalse啟用重新排序
modelstring-重新排序模型（例如：cohere/rerank-english-v3.0）
top_ninteger5重新排序後的結果數量

端到端範例​

1. 匯入文件​

首先，使用 /rag/ingest 端點匯入文件：

Step 1: Ingest
curl -X POST "http://localhost:4000/v1/rag/ingest" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{
        \"file\": {
            \"filename\": \"company_docs.txt\",
            \"content\": \"$(base64 -i company_docs.txt)\",
            \"content_type\": \"text/plain\"
        },
        \"ingest_options\": {
            \"vector_store\": {
                \"custom_llm_provider\": \"openai\"
            }
        }
    }"

回應：

{
  "id": "ingest_abc123",
  "status": "completed",
  "vector_store_id": "vs_xyz789",
  "file_id": "file-123"
}

2. 使用 RAG 查詢​

現在查詢已匯入的文件：

Step 2: Query
curl -X POST "http://localhost:4000/v1/rag/query" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "gpt-5.6-luna",
        "messages": [
            {"role": "user", "content": "What products does the company offer?"}
        ],
        "retrieval_config": {
            "vector_store_id": "vs_xyz789",
            "custom_llm_provider": "openai",
            "top_k": 5
        }
    }'

回應：

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "model": "gpt-5.6-luna",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Based on the company documents, the company offers..."
      },
      "finish_reason": "stop"
    }
  ]
}

提供者範例​

Bedrock​

RAG Query with Bedrock
curl -X POST "http://localhost:4000/v1/rag/query" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "bedrock/us.anthropic.claude-sonnet-5",
        "messages": [{"role": "user", "content": "What is LiteLLM?"}],
        "retrieval_config": {
            "vector_store_id": "KNOWLEDGE_BASE_ID",
            "custom_llm_provider": "bedrock",
            "top_k": 5
        }
    }'

Vertex AI​

RAG Query with Vertex AI
curl -X POST "http://localhost:4000/v1/rag/query" \
    -H "Authorization: Bearer $LITELLM_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "vertex_ai/gemini-3.1-pro-preview",
        "messages": [{"role": "user", "content": "What is LiteLLM?"}],
        "retrieval_config": {
            "vector_store_id": "your-corpus-id",
            "custom_llm_provider": "vertex_ai",
            "top_k": 5
        }
    }'

Python SDK​

Using litellm.aquery()
import litellm

response = await litellm.aquery(
    model="gpt-5.6-luna",
    messages=[{"role": "user", "content": "What is LiteLLM?"}],
    retrieval_config={
        "vector_store_id": "vs_abc123",
        "custom_llm_provider": "openai",
        "top_k": 5,
    },
    rerank={
        "enabled": True,
        "model": "cohere/rerank-english-v3.0",
        "top_n": 3,
    },
)

print(response.choices[0].message.content)
LiteLLM Enterprise
為正式環境打造的 SSO/SAML、稽核記錄、支出追蹤、多團隊管理與防護欄。
深入瞭解 →