跳至主要內容

「思考」/「推理內容」

資訊

需要 LiteLLM v1.63.0+

支援的提供者：

  • Deepseek (deepseek/)
  • Anthropic API (anthropic/)
  • Bedrock（Anthropic + Deepseek + GPT-OSS）(bedrock/)
  • OpenAI Responses API (openai/responses/)
  • Vertex AI（Anthropic）(vertexai/)
  • OpenRouter (openrouter/)
  • XAI (xai/)
  • Google AI Studio (google/)
  • Vertex AI (vertex_ai/)
  • Perplexity (perplexity/)
  • Mistral AI（Magistral models）(mistral/)
  • Groq (groq/)

LiteLLM 會將回應中的 reasoning_content 和助理訊息中的 thinking_blocks 標準化。

Example response from litellm
"message": {
    ...
    "reasoning_content": "The capital of France is Paris.",
    "thinking_blocks": [ # only returned for Anthropic models
        {
            "type": "thinking",
            "thinking": "The capital of France is Paris.",
            "signature": "EqoBCkgIARABGAIiQL2UoU0b1OHYi+..."
        }
    ]
}

快速開始​

from litellm import completion
import os 

os.environ["ANTHROPIC_API_KEY"] = ""

response = completion(
  model="anthropic/claude-3-7-sonnet-20250219",
  messages=[
    {"role": "user", "content": "What is the capital of France?"},
  ],
  reasoning_effort="low", 
)
print(response.choices[0].message.content)

預期回應

{
    "id": "3b66124d79a708e10c603496b363574c",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "message": {
                "content": " won the FIFA World Cup in 2022.",
                "role": "assistant",
                "tool_calls": null,
                "function_call": null
            }
        }
    ],
    "created": 1723323084,
    "model": "deepseek/deepseek-chat",
    "object": "chat.completion",
    "system_fingerprint": "fp_7e0991cad4",
    "usage": {
        "completion_tokens": 12,
        "prompt_tokens": 16,
        "total_tokens": 28,
    },
    "service_tier": null
}

使用 thinking 的工具呼叫​

以下說明如何搭配工具呼叫使用 Anthropic 的 thinking 區塊。

重要：OpenAI 相容 API 的限制​

相容性注意事項

Anthropic 的 extended thinking 搭配工具呼叫，與 OpenAI 相容 API 用戶端 並非完全相容。這是因為 OpenAI 與 Anthropic 在多輪對話中處理推理的架構有根本差異。

當使用啟用 thinking 且有工具呼叫的 Anthropic 模型時，您 必須包含 thinking_blocks，也就是前一次助理回應中的內容，並在將工具結果送回時一併傳入。若未這麼做，將會產生 400 Bad Request 錯誤。

OpenAI 與 Anthropic 架構：

提供者API 架構推理儲存多輪處理
OpenAI (o1, o3)Responses API（有狀態）伺服器端伺服器會在內部儲存推理；用戶端會傳送 previous_response_id
Anthropic (Claude)Messages API（無狀態）用戶端用戶端必須儲存並在每次請求時重新傳送 thinking_blocks
  1. OpenAI 的 Chat Completions 規格 沒有欄位可用於 thinking_blocks
  2. OpenAI 相容用戶端（LibreChat、Open WebUI、Vercel AI SDK 等）會 忽略 回應中的 thinking_blocks 欄位
  3. 這些用戶端在重建下一輪的助理訊息時，thinking blocks 會遺失
  4. Anthropic 會拒絕該請求，因為助理訊息不是以 thinking block 開頭
LiteLLM supports thinking_blocks

LiteLLM 的 completion() API 確實支援 在助理訊息中傳送 thinking_blocks。如果您是直接使用 LiteLLM（不是透過 OpenAI 相容用戶端），就可以保留並重新傳送 thinking_blocks，一切都會正常運作。

解決方案：

  1. 使用 LiteLLM 內建的因應措施（建議）：設定 litellm.modify_params = True，當 thinking 缺失時，LiteLLM 會自動透過移除 thinking_blocks 參數來處理此不相容問題（見下方）
  2. 給用戶端開發者：明確處理並重新傳送 thinking_blocks 欄位（見下方範例）
  3. 在使用工具且搭配不支援 thinking_blocks 的 OpenAI 相容用戶端時，停用 extended thinking
  4. 直接使用 Anthropic 的原生 API，而不是 OpenAI 相容端點

LiteLLM 內建因應措施​

當設定 modify_params=True 時，LiteLLM 可以自動處理此不相容問題。如果用戶端送出啟用 thinking 的請求，但帶有 tool_calls 的助理訊息缺少 thinking_blocks，LiteLLM 會自動在該輪移除 thinking 參數，以避免錯誤。

import litellm

# Enable automatic parameter modification
litellm.modify_params = True

# Now this will work even if thinking_blocks are missing from the assistant message
response = litellm.completion(
    model="anthropic/claude-sonnet-4-20250514",
    thinking={"type": "enabled", "budget_tokens": 1024},
    tools=[...],
    messages=[
        {"role": "user", "content": "What's the weather in Madrid?"},
        {
            "role": "assistant",
            "tool_calls": [{"id": "call_123", "type": "function", "function": {"name": "get_weather", "arguments": '{"city": "Madrid"}'}}]
            # Note: thinking_blocks is missing here - LiteLLM will handle it
        },
        {"role": "tool", "tool_call_id": "call_123", "content": "22°C sunny"}
    ]
)
資訊

當 modify_params=True 且 LiteLLM 移除 thinking 參數時，模型在該輪 不會 使用 extended thinking。對話會正常繼續，但該次回應不會有推理。

正確包含 thinking_blocks 的方式：

# After receiving a response with tool_calls, include thinking_blocks when sending back:
assistant_message = {
    "role": "assistant",
    "content": response.choices[0].message.content,
    "tool_calls": [...],
    "thinking_blocks": response.choices[0].message.thinking_blocks  # ← Required!
}
litellm._turn_on_debug()
litellm.modify_params = True
model = "anthropic/claude-3-7-sonnet-20250219" # works across Anthropic, Bedrock, Vertex AI
# Step 1: send the conversation and available functions to the model
messages = [
    {
        "role": "user",
        "content": "What's the weather like in San Francisco, Tokyo, and Paris? - give me 3 responses",
    }
]
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                    },
                },
                "required": ["location"],
            },
        },
    }
]
response = litellm.completion(
    model=model,
    messages=messages,
    tools=tools,
    tool_choice="auto",  # auto is default, but we'll be explicit
    reasoning_effort="low",
)
print("Response\n", response)
response_message = response.choices[0].message
tool_calls = response_message.tool_calls

print("Expecting there to be 3 tool calls")
assert (
    len(tool_calls) > 0
)  # this has to call the function for SF, Tokyo and paris

# Step 2: check if the model wanted to call a function
print(f"tool_calls: {tool_calls}")
if tool_calls:
    # Step 3: call the function
    # Note: the JSON response may not always be valid; be sure to handle errors
    available_functions = {
        "get_current_weather": get_current_weather,
    }  # only one function in this example, but you can have multiple
    messages.append(
        response_message
    )  # extend conversation with assistant's reply
    print("Response message\n", response_message)
    # Step 4: send the info for each function call and function response to the model
    for tool_call in tool_calls:
        function_name = tool_call.function.name
        if function_name not in available_functions:
            # the model called a function that does not exist in available_functions - don't try calling anything
            return
        function_to_call = available_functions[function_name]
        function_args = json.loads(tool_call.function.arguments)
        function_response = function_to_call(
            location=function_args.get("location"),
            unit=function_args.get("unit"),
        )
        messages.append(
            {
                "tool_call_id": tool_call.id,
                "role": "tool",
                "name": function_name,
                "content": function_response,
            }
        )  # extend conversation with function response
    print(f"messages: {messages}")
    second_response = litellm.completion(
        model=model,
        messages=messages,
        seed=22,
        reasoning_effort="low",
        # tools=tools,
        drop_params=True,
    )  # get a new response from the model where it can see the function response
    print("second response\n", second_response)

在 Anthropic + Deepseek 模型之間切換​

將 drop_params=True 設為在從 Anthropic 切換到 Deepseek 模型時移除 'thinking' 區塊。可在這裡提出對此做法的改進建議。

litellm.drop_params = True # 👈 EITHER GLOBALLY or per request

# or per request
## Anthropic
response = litellm.completion(
  model="anthropic/claude-3-7-sonnet-20250219",
  messages=[{"role": "user", "content": "What is the capital of France?"}],
  reasoning_effort="low",
  drop_params=True,
)

## Deepseek
response = litellm.completion(
  model="deepseek/deepseek-chat",
  messages=[{"role": "user", "content": "What is the capital of France?"}],
  reasoning_effort="low",
  drop_params=True,
)

規格​

這些欄位可透過 response.choices[0].message.reasoning_content 和 response.choices[0].message.thinking_blocks 存取。

  • reasoning_content - str：來自模型的推理內容。所有提供者皆會回傳。
  • thinking_blocks - Optional[List[Dict[str, str]]]：來自模型的 thinking blocks 清單。僅 Anthropic 模型會回傳。
    • type - str：thinking block 的類型。
    • thinking - str：來自模型的 thinking。
    • signature - str：來自模型的 signature delta。

將 thinking 傳給 Anthropic 模型​

您也可以將 thinking 參數傳給 Anthropic 模型。

response = litellm.completion(
  model="anthropic/claude-3-7-sonnet-20250219",
  messages=[{"role": "user", "content": "What is the capital of France?"}],
  thinking={"type": "enabled", "budget_tokens": 1024},
)

檢查模型是否支援推理​

使用 litellm.supports_reasoning(model="") -> 若模型支援推理則回傳 True，否則回傳 False。

litellm.supports_reasoning() usage
import litellm 

# Example models that support reasoning
assert litellm.supports_reasoning(model="anthropic/claude-3-7-sonnet-20250219") == True
assert litellm.supports_reasoning(model="deepseek/deepseek-chat") == True 

# Example models that do not support reasoning
assert litellm.supports_reasoning(model="openai/gpt-3.5-turbo") == False
gpt-5.4: reasoning_effort + function tools

當 gpt-5.4+ 對 litellm.completion() 的請求同時包含 reasoning_effort 和 tools 時，LiteLLM 會 自動 透過 Responses API bridge 來路由該請求。這同時適用於 OpenAI (openai/gpt-5.4) 與 Azure (azure/gpt-5.4) 提供者 — 無需額外設定。

您也可以透過 openai/responses/gpt-5.4 或 azure/responses/gpt-5.4 明確路由。詳情請參閱 Responses API Bridge。

Azure 自訂部署名稱： 自動路由依賴部署名稱符合 gpt-5.4* 模式。若您使用自訂部署名稱（例如 "my-reasoning-model"），請透過下列方式啟用路由：

SDK：

litellm.completion(model="azure/responses/my-reasoning-model", ...)

Proxy 設定：

model_list:
  - model_name: my-reasoning-model
    litellm_params:
      model: azure/my-reasoning-model
    model_info:
      mode: responses

OpenAI Responses API - 自動摘要控制​

使用 OpenAI Responses API 模型（例如 gpt-5）並透過 /chat/completions 搭配 reasoning_effort 時，您可以控制是否要將 summary="detailed" 自動加入 reasoning 參數。

啟用自動摘要​

您可以透過兩種方式啟用自動 summary="detailed"：

import litellm

# Enable auto-summary globally
litellm.reasoning_auto_summary = True

response = litellm.completion(
    model="openai/responses/gpt-5-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    reasoning_effort="low",  # Will automatically add summary="detailed"
)

若要進行細緻控制，請將 reasoning_effort 以字典形式傳入：

response = litellm.completion(
    model="openai/responses/gpt-5-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    reasoning_effort={"effort": "low", "summary": "detailed"},  # Explicit control
)

透過 /v1/messages Adapter 保留摘要​

使用 Anthropic /v1/messages adapter 將請求路由到非 Claude 模型（例如 openai/gpt-5.1）時，thinking.summary 值會被保留並轉送至下游提供者。範例如下：

import litellm

response = await litellm.anthropic.messages.acreate(
    model="openai/gpt-5.1",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=8096,
    thinking={"type": "enabled", "budget_tokens": 5000, "summary": "concise"},
)
# The summary="concise" is preserved when routing to OpenAI's Responses API

為 /v1/messages Adapter 啟用預設摘要注入​

當 Anthropic /v1/messages adapter 將非 Claude 模型的 thinking 參數轉換為 OpenAI reasoning_effort 時，您可以透過 summary="detailed" 旗標選擇啟用自動 reasoning_auto_summary 注入。這可確保推理文字會在回應中傳回（與 Anthropic thinking 行為一致）。

若要啟用此預設注入，請使用 reasoning_auto_summary 旗標：

import litellm

# Enable default summary="detailed" injection
litellm.reasoning_auto_summary = True

response = await litellm.anthropic.messages.acreate(
    model="openai/gpt-5.1",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=8096,
    thinking={"type": "enabled", "budget_tokens": 5000},
)
# summary="detailed" will be automatically added to reasoning_effort
資訊

此旗標只會在沒有使用者提供的摘要時，影響 summary="detailed" 的自動注入。若您明確傳入 thinking.summary（例如 "concise" 或 "auto"），無論此旗標為何，您的值都會一律保留。

🚅
LiteLLM Enterprise
為正式環境打造的 SSO/SAML、稽核記錄、支出追蹤、多團隊管理與防護欄。
深入瞭解 →