跳至主要內容

Bedrock 匯入模型

Bedrock 匯入模型（Deepseek、Deepseek R1、Qwen、OpenAI 相容模型）

Deepseek R1​

這是一條獨立路由，因為聊天範本不同。

屬性詳細資訊
提供者路由bedrock/deepseek_r1/{model_arn}
提供者文件Bedrock Imported Models, Deepseek Bedrock Imported Model
from litellm import completion
import os

response = completion(
    model="bedrock/deepseek_r1/arn:aws:bedrock:us-east-1:086734376398:imported-model/r4c4kewx2s0n",  # bedrock/deepseek_r1/{your-model-arn}
    messages=[{"role": "user", "content": "Tell me a joke"}],
)

Deepseek（非 R1）​

屬性詳細資訊
提供者路由bedrock/llama/{model_arn}
提供者文件Bedrock Imported Models, Deepseek Bedrock Imported Model

使用此路由可呼叫遵循 llama Invoke Request / Response 規格的 Bedrock 匯入模型

from litellm import completion
import os

response = completion(
    model="bedrock/llama/arn:aws:bedrock:us-east-1:086734376398:imported-model/r4c4kewx2s0n",  # bedrock/llama/{your-model-arn}
    messages=[{"role": "user", "content": "Tell me a joke"}],
)

Qwen3 匯入模型​

屬性詳細資訊
提供者路由bedrock/qwen3/{model_arn}
提供者文件Bedrock Imported Models, Qwen3 Models
from litellm import completion
import os

response = completion(
    model="bedrock/qwen3/arn:aws:bedrock:us-east-1:086734376398:imported-model/your-qwen3-model",  # bedrock/qwen3/{your-model-arn}
    messages=[{"role": "user", "content": "Tell me a joke"}],
    max_tokens=100,
    temperature=0.7
)

Qwen2 匯入模型​

屬性詳細資訊
提供者路由bedrock/qwen2/{model_arn}
提供者文件Bedrock Imported Models
NoteQwen2 與 Qwen3 架構大致相似。主要差異在回應格式：Qwen2 使用「text」欄位，而 Qwen3 使用「generation」欄位。
from litellm import completion
import os

response = completion(
    model="bedrock/qwen2/arn:aws:bedrock:us-east-1:086734376398:imported-model/your-qwen2-model",  # bedrock/qwen2/{your-model-arn}
    messages=[{"role": "user", "content": "Tell me a joke"}],
    max_tokens=100,
    temperature=0.7
)

OpenAI 相容匯入模型（Qwen 2.5 VL 等）​

使用此路由可處理遵循 OpenAI Chat Completions API 規格 的 Bedrock 匯入模型。這包括像 Qwen 2.5 VL 這類接受 OpenAI 格式訊息的模型，並支援 vision（圖片）、tool calling 及其他 OpenAI 功能。

屬性詳細資訊
提供者路由bedrock/openai/{model_arn}
提供者文件Bedrock Imported Models
Supported FeaturesVision（圖片）、tool calling、串流、system 訊息

LiteLLMSDK 使用方式​

基本使用

from litellm import completion

response = completion(
    model="bedrock/openai/arn:aws:bedrock:us-east-1:046319184608:imported-model/0m2lasirsp6z",  # bedrock/openai/{your-model-arn}
    messages=[{"role": "user", "content": "Tell me a joke"}],
    max_tokens=300,
    temperature=0.5
)

搭配 Vision（圖片）

import base64
from litellm import completion

# Load and encode image
with open("image.jpg", "rb") as f:
    image_base64 = base64.b64encode(f.read()).decode("utf-8")

response = completion(
    model="bedrock/openai/arn:aws:bedrock:us-east-1:046319184608:imported-model/0m2lasirsp6z",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant that can analyze images."
        },
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}
                }
            ]
        }
    ],
    max_tokens=300,
    temperature=0.5
)

比較多張圖片

import base64
from litellm import completion

# Load images
with open("image1.jpg", "rb") as f:
    image1_base64 = base64.b64encode(f.read()).decode("utf-8")
with open("image2.jpg", "rb") as f:
    image2_base64 = base64.b64encode(f.read()).decode("utf-8")

response = completion(
    model="bedrock/openai/arn:aws:bedrock:us-east-1:046319184608:imported-model/0m2lasirsp6z",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant that can analyze images."
        },
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Spot the difference between these two images?"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{image1_base64}"}
                },
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{image2_base64}"}
                }
            ]
        }
    ],
    max_tokens=300,
    temperature=0.5
)

LiteLLM Proxy 使用方式（AI Gateway）​

1. 新增至設定

model_list:
    - model_name: qwen-25vl-72b
      litellm_params:
        model: bedrock/openai/arn:aws:bedrock:us-east-1:046319184608:imported-model/0m2lasirsp6z

2. 啟動 proxy

litellm --config /path/to/config.yaml

# RUNNING at http://0.0.0.0:4000

3. 測試它！

基本文字請求：

curl --location 'http://0.0.0.0:4000/chat/completions' \
      --header 'Authorization: Bearer sk-1234' \
      --header 'Content-Type: application/json' \
      --data '{
            "model": "qwen-25vl-72b",
            "messages": [
                {
                    "role": "user",
                    "content": "what llm are you"
                }
            ],
            "max_tokens": 300
        }'

搭配 vision（圖片）：

curl --location 'http://0.0.0.0:4000/chat/completions' \
      --header 'Authorization: Bearer sk-1234' \
      --header 'Content-Type: application/json' \
      --data '{
            "model": "qwen-25vl-72b",
            "messages": [
                {
                    "role": "system",
                    "content": "You are a helpful assistant that can analyze images."
                },
                {
                    "role": "user",
                    "content": [
                        {"type": "text", "text": "What is in this image?"},
                        {
                            "type": "image_url",
                            "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQSkZ..."}
                        }
                    ]
                }
            ],
            "max_tokens": 300,
            "temperature": 0.5
        }'

Moonshot Kimi K2 Thinking​

Moonshot AI 的 Kimi K2 Thinking 模型現已於 Amazon Bedrock 上提供。此模型具備進階推理能力，並可自動擷取推理內容。

屬性詳細資訊
提供者路由bedrock/moonshot.kimi-k2-thinking, bedrock/invoke/moonshot.kimi-k2-thinking
提供者文件AWS Bedrock Moonshot Announcement ↗
支援的參數temperature, max_tokens, top_p, stream, tools, tool_choice
Special Features推理內容擷取、tool calling

支援功能​

  • 推理內容擷取：自動擷取 <reasoning> 標籤，並將其以 reasoning_content 回傳（類似 OpenAI 的 o1 模型）
  • Tool Calling：完整支援 function/tool calling 與 tool 回應
  • Streaming：支援串流與非串流回應
  • System Messages：支援 system 訊息

基本使用​

Moonshot Kimi K2 SDK Usage
from litellm import completion
import os

os.environ["AWS_ACCESS_KEY_ID"] = "your-aws-access-key"
os.environ["AWS_SECRET_ACCESS_KEY"] = "your-aws-secret-key"
os.environ["AWS_REGION_NAME"] = "us-west-2"  # or your preferred region

# Basic completion
response = completion(
    model="bedrock/moonshot.kimi-k2-thinking",  # or bedrock/invoke/moonshot.kimi-k2-thinking
    messages=[
        {"role": "user", "content": "What is 2+2? Think step by step."}
    ],
    temperature=0.7,
    max_tokens=200
)

print(response.choices[0].message.content)

# Access reasoning content if present
if response.choices[0].message.reasoning_content:
    print("Reasoning:", response.choices[0].message.reasoning_content)

Tool Calling 範例​

Kimi K2 with Tool Calling
from litellm import completion
import os

os.environ["AWS_ACCESS_KEY_ID"] = "your-aws-access-key"
os.environ["AWS_SECRET_ACCESS_KEY"] = "your-aws-secret-key"
os.environ["AWS_REGION_NAME"] = "us-west-2"

# Tool calling example
response = completion(
    model="bedrock/moonshot.kimi-k2-thinking",
    messages=[
        {"role": "user", "content": "What's the weather in Tokyo?"}
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather in a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city name"
                        }
                    },
                    "required": ["location"]
                }
            }
        }
    ]
)

if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Tool called: {tool_call.function.name}")
    print(f"Arguments: {tool_call.function.arguments}")

串流範例​

Kimi K2 Streaming
from litellm import completion
import os

os.environ["AWS_ACCESS_KEY_ID"] = "your-aws-access-key"
os.environ["AWS_SECRET_ACCESS_KEY"] = "your-aws-secret-key"
os.environ["AWS_REGION_NAME"] = "us-west-2"

response = completion(
    model="bedrock/moonshot.kimi-k2-thinking",
    messages=[
        {"role": "user", "content": "Explain quantum computing in simple terms."}
    ],
    stream=True,
    temperature=0.7
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
    
    # Check for reasoning content in streaming
    if hasattr(chunk.choices[0].delta, 'reasoning_content') and chunk.choices[0].delta.reasoning_content:
        print(f"\n[Reasoning: {chunk.choices[0].delta.reasoning_content}]")

支援參數​

參數類型說明Supported
temperaturefloat (0-1)控制輸出的隨機性✅
max_tokensinteger可生成的最大 token 數✅
top_pfloat核心採樣參數✅
streamboolean啟用串流回應✅
toolsarraytool/function 定義✅
tool_choicestring/objecttool 選擇規格✅
stoparray停止序列❌（Bedrock 不支援）
🚅
LiteLLM Enterprise
為正式環境打造的 SSO/SAML、稽核記錄、支出追蹤、多團隊管理與防護欄。
深入瞭解 →