Bedrock 知識庫

AWS Bedrock Knowledge Bases 可讓您將 LLM 連接至您組織的資料，讓模型擷取並參照與您業務相關的資訊。

屬性	詳細資訊
說明	Bedrock Knowledge Bases 將您的資料連接至 LLM，讓它們能在回應中擷取並參照您組織的資訊。
LiteLLM 提供者路由	bedrock 於 litellm vector_store_registry 中
提供者文件	AWS Bedrock Knowledge Bases ↗

快速開始

LiteLLM Python SDK

Example using LiteLLM Python SDK

import os
import litellm

from litellm.vector_stores.vector_store_registry import VectorStoreRegistry, LiteLLM_ManagedVectorStore

# Init vector store registry with your Bedrock Knowledge Base
litellm.vector_store_registry = VectorStoreRegistry(
    vector_stores=[
        LiteLLM_ManagedVectorStore(
            vector_store_id="YOUR_KNOWLEDGE_BASE_ID",  # KB ID from AWS Bedrock
            custom_llm_provider="bedrock"
        )
    ]
)

# Make a completion request using your Knowledge Base
response = await litellm.acompletion(
    model="anthropic/claude-sonnet-5", 
    messages=[{"role": "user", "content": "What does our company policy say about remote work?"}],
    tools=[
        {
            "type": "file_search",
            "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"]
        }
    ],
)

print(response.choices[0].message.content)

LiteLLM Proxy

1. 設定您的 vector_store_registry

config.yaml

model_list:
  - model_name: claude-sonnet-5
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY

vector_store_registry:
  - vector_store_name: "bedrock-company-docs"
    litellm_params:
      vector_store_id: "YOUR_KNOWLEDGE_BASE_ID"
      custom_llm_provider: "bedrock"
      vector_store_description: "Bedrock Knowledge Base for company documents"
      vector_store_metadata:
        source: "Company internal documentation"

LiteLLM UI

在 LiteLLM UI 中，前往 Experimental > Vector Stores > Create Vector Store。在此頁面中，您可以建立具有名稱、vector store id 和憑證的 vector store。

2. 使用 vector_store_ids 參數發出請求

Curl

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [{"role": "user", "content": "What does our company policy say about remote work?"}],
    "tools": [
        {
            "type": "file_search",
            "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"]
        }
    ]
  }'

OpenAI Python SDK

from openai import OpenAI

# Initialize client with your LiteLLM proxy URL
client = OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-api-key"
)

# Make a completion request with vector_store_ids parameter
response = client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": "What does our company policy say about remote work?"}],
    tools=[
        {
            "type": "file_search",
            "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"]
        }
    ]
)

print(response.choices[0].message.content)

篩選結果

依中繼資料屬性篩選。

運算子（OpenAI 樣式，自動轉譯）：

• eq, ne, gt, gte, lt, lte, in, nin

AWS 運算子（直接使用）：

• equals, notEquals, greaterThan, greaterThanOrEquals, lessThan, lessThanOrEquals, in, notIn, startsWith, listContains, stringContains

單一篩選器

response = await litellm.acompletion(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "What are the latest updates?"}],
    tools=[{
        "type": "file_search",
        "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"],
        "filters": {
            "key": "category",
            "value": "updates",
            "operator": "eq"
        }
    }]
)

AND

response = await litellm.acompletion(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "What are the policies?"}],
    tools=[{
        "type": "file_search",
        "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"],
        "filters": {
            "and": [
                {"key": "category", "value": "policy", "operator": "eq"},
                {"key": "year", "value": 2024, "operator": "gte"}
            ]
        }
    }]
)

OR

response = await litellm.acompletion(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Show me technical docs"}],
    tools=[{
        "type": "file_search",
        "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"],
        "filters": {
            "or": [
                {"key": "category", "value": "api", "operator": "eq"},
                {"key": "category", "value": "sdk", "operator": "eq"}
            ]
        }
    }]
)

AWS 運算子

response = await litellm.acompletion(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Find docs"}],
    tools=[{
        "type": "file_search",
        "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"],
        "filters": {
            "and": [
                {"key": "title", "value": "Guide", "operator": "stringContains"},
                {"key": "tags", "value": "important", "operator": "listContains"}
            ]
        }
    }]
)

Proxy

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [{"role": "user", "content": "What are our policies?"}],
    "tools": [{
        "type": "file_search",
        "vector_store_ids": ["YOUR_KNOWLEDGE_BASE_ID"],
        "filters": {
            "and": [
                {"key": "department", "value": "engineering", "operator": "eq"},
                {"key": "type", "value": "policy", "operator": "eq"}
            ]
        }
    }]
  }'

將結果限制為某位使用者

具有存取控制的 Knowledge Bases（Kendra GenAI index 或具有文件層級 ACL 的資料來源）只會回傳特定使用者可見的區塊。將該身分作為 Bedrock 的 userContext 傳入，LiteLLM 會在 Retrieve 請求中轉送它。該值會原樣送出，因此 Bedrock 會驗證它：userId 必須是字串。LiteLLM 不會將該身分與 proxy key 進行比對，因此只有您信任其可為使用者命名的呼叫端，才能搜尋具有 ACL 的 store。

LiteLLM Python SDK

import litellm

response = litellm.vector_stores.search(
    vector_store_id="YOUR_KNOWLEDGE_BASE_ID",
    custom_llm_provider="bedrock",
    query="What does our company policy say about remote work?",
    extra_body={"userContext": {"userId": "alice@example.com"}},
)

Proxy (OpenAI SDK)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:4000", api_key="your-litellm-api-key")

response = client.vector_stores.search(
    vector_store_id="YOUR_KNOWLEDGE_BASE_ID",
    query="What does our company policy say about remote work?",
    extra_body={"userContext": {"userId": "alice@example.com"}},
)

Proxy (curl)

curl http://localhost:4000/v1/vector_stores/YOUR_KNOWLEDGE_BASE_ID/search \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "query": "What does our company policy say about remote work?",
    "userContext": {"userId": "alice@example.com"}
  }'

Proxy (config.yaml)

在 store 上設定 user_context，即可將單一身分套用到對它的每次搜尋。於請求中送出的 userContext 會覆寫它。

vector_store_registry:
  - vector_store_name: "bedrock-company-docs"
    litellm_params:
      vector_store_id: "YOUR_KNOWLEDGE_BASE_ID"
      custom_llm_provider: "bedrock"
      user_context:
        userId: "service-account@example.com"

存取搜尋結果

了解如何在回應中存取 vector store 搜尋結果：

• 存取搜尋結果（非串流與串流）

延伸閱讀

Vector Stores：

• Always on Vector Stores
• 在 litellm proxy 上列出可用的 vector stores
• LiteLLM Vector Stores 的運作方式