Amazon Bedrock Mantle

Amazon Bedrock Mantle 是 Amazon Bedrock 的分散式推論引擎（Project Mantle），可為 Bedrock 托管的模型提供 OpenAI 相容 API。

請使用此提供者，以正確的 AWS Bedrock 定價 而非 OpenAI 定價來呼叫 Bedrock Mantle 模型。

提示

我們支援所有 Bedrock Mantle 模型，只要在傳送 litellm 請求時將 model=bedrock_mantle/<model-id> 設為前綴即可

Claude Mythos

Claude Mythos (anthropic.claude-mythos-preview) 可在 Bedrock Mantle 上使用，具備 1M token 輸入上下文、128K 輸出，以及推理、視覺和工具使用支援。

請使用 bedrock_mantle/ 路由前綴搭配標準 AWS 憑證。

/messages

SDK

import asyncio
import litellm
import os

os.environ['AWS_ACCESS_KEY_ID'] = "your-aws-access-key"
os.environ['AWS_SECRET_ACCESS_KEY'] = "your-aws-secret-key"
os.environ['AWS_REGION_NAME'] = "us-east-1"

async def main():
    response = await litellm.anthropic_messages(
        model="bedrock_mantle/anthropic.claude-mythos-preview",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Explain quantum entanglement simply."}],
    )
    print(response)

asyncio.run(main())

AI Gateway

1. 新增至 config.yaml

model_list:
  - model_name: claude-mythos
    litellm_params:
      model: bedrock_mantle/anthropic.claude-mythos-preview
      aws_region_name: us-east-1

2. 啟動 LiteLLM AI Gateway

litellm --config /path/to/config.yaml

3. 透過 curl 呼叫 /v1/messages

curl -X POST http://0.0.0.0:4000/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "claude-mythos",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Explain quantum entanglement simply."}
    ]
  }'

/chat/completions

SDK

from litellm import completion
import os

os.environ['AWS_ACCESS_KEY_ID'] = "your-aws-access-key"
os.environ['AWS_SECRET_ACCESS_KEY'] = "your-aws-secret-key"
os.environ['AWS_REGION_NAME'] = "us-east-1"

response = completion(
    model="bedrock_mantle/anthropic.claude-mythos-preview",
    messages=[{"role": "user", "content": "Explain quantum entanglement simply."}],
)
print(response)

AI Gateway

1. 新增至 config.yaml

model_list:
  - model_name: claude-mythos
    litellm_params:
      model: bedrock_mantle/anthropic.claude-mythos-preview
      aws_region_name: us-east-1

2. 啟動 LiteLLM AI Gateway

litellm --config /path/to/config.yaml

3. 透過 curl 呼叫 /v1/chat/completions

curl -X POST http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "claude-mythos",
    "messages": [
      {"role": "user", "content": "Explain quantum entanglement simply."}
    ]
  }'

OpenAI 模型 (GPT-5.4 / GPT-5.5)

/responses

SDK

import litellm
import os

os.environ['BEDROCK_MANTLE_API_KEY'] = "your-bedrock-api-key"
os.environ['BEDROCK_MANTLE_REGION'] = "us-east-2"

response = litellm.responses(
    model="bedrock_mantle/openai.gpt-5.5",
    input="Hello! How can you help me today?",
)
print(response)

串流

import litellm
import os

os.environ['BEDROCK_MANTLE_API_KEY'] = "your-bedrock-api-key"

response = litellm.responses(
    model="bedrock_mantle/openai.gpt-5.5",
    input="Tell me a three sentence bedtime story about a unicorn.",
    stream=True,
)

for event in response:
    print(event)

AI Gateway

1. 新增至 config.yaml

model_list:
  - model_name: gpt-5.5-mantle
    litellm_params:
      model: bedrock_mantle/openai.gpt-5.5
      api_key: os.environ/BEDROCK_MANTLE_API_KEY
      api_base: https://bedrock-mantle.us-east-2.api.aws/v1

2. 啟動 LiteLLM AI Gateway

litellm --config /path/to/config.yaml

3. 透過 curl 呼叫 /v1/responses

curl -X POST http://0.0.0.0:4000/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "gpt-5.5-mantle",
    "input": "Hello! How can you help me today?"
  }'

4. 或使用 OpenAI SDK

from openai import OpenAI

client = OpenAI(
    api_key="sk-1234",
    base_url="http://0.0.0.0:4000",
)

response = client.responses.create(
    model="gpt-5.5-mantle",
    input="Hello! How can you help me today?",
)
print(response)

API 金鑰

# env variable
os.environ['BEDROCK_MANTLE_API_KEY'] = "your-aws-bedrock-api-key"

# optional: override region (defaults to us-east-1)
os.environ['BEDROCK_MANTLE_REGION'] = "us-east-1"  # or use AWS_REGION

支援的模型

模型	端點	上下文視窗	輸入（每 1M tokens）	輸出（每 1M tokens）
openai.gpt-5.5	/responses	272K	$5.50	$33.00
openai.gpt-5.4	/responses	272K	$2.75	$16.50
openai.gpt-oss-120b	/chat/completions	131K	$0.15	$0.60
openai.gpt-oss-20b	/chat/completions	131K	$0.075	$0.30
openai.gpt-oss-safeguard-120b	/chat/completions	131K	$0.15	$0.60
openai.gpt-oss-safeguard-20b	/chat/completions	131K	$0.075	$0.30

範例用法

SDK

from litellm import completion
import os

os.environ['BEDROCK_MANTLE_API_KEY'] = "your-bedrock-api-key"

response = completion(
    model="bedrock_mantle/openai.gpt-oss-120b",
    messages=[{"role": "user", "content": "hello from litellm"}],
)
print(response)

Streaming

from litellm import completion
import os

os.environ['BEDROCK_MANTLE_API_KEY'] = "your-bedrock-api-key"

response = completion(
    model="bedrock_mantle/openai.gpt-oss-120b",
    messages=[{"role": "user", "content": "hello from litellm"}],
    stream=True,
)

for chunk in response:
    print(chunk)

Async

import asyncio
from litellm import acompletion
import os

os.environ['BEDROCK_MANTLE_API_KEY'] = "your-bedrock-api-key"

async def main():
    response = await acompletion(
        model="bedrock_mantle/openai.gpt-oss-120b",
        messages=[{"role": "user", "content": "hello from litellm"}],
    )
    print(response)

asyncio.run(main())

區域設定

API 基礎 URL 為 https://bedrock-mantle.{region}.api.aws/v1。區域會依下列順序解析：

1. BEDROCK_MANTLE_REGION 環境變數
2. AWS_REGION 環境變數
3. 預設：us-east-1

支援的區域： us-east-1, us-east-2, us-west-2, eu-west-1, eu-west-2, eu-central-1, eu-south-1, eu-north-1, ap-northeast-1, ap-south-1, ap-southeast-3, sa-east-1

import os
os.environ['BEDROCK_MANTLE_REGION'] = "eu-west-1"

# or pass api_base directly
response = completion(
    model="bedrock_mantle/openai.gpt-oss-120b",
    messages=[{"role": "user", "content": "hello"}],
    api_base="https://bedrock-mantle.eu-west-1.api.aws/v1",
)

搭配 LiteLLM Proxy 使用

1. 在 config.yaml 上設定 Bedrock Mantle 模型

model_list:
  - model_name: gpt-5.5-mantle
    litellm_params:
      model: bedrock_mantle/openai.gpt-5.5
      api_key: os.environ/BEDROCK_MANTLE_API_KEY
      api_base: "https://bedrock-mantle.us-east-2.api.aws/v1"

  - model_name: gpt-oss-120b
    litellm_params:
      model: bedrock_mantle/openai.gpt-oss-120b
      api_key: os.environ/BEDROCK_MANTLE_API_KEY
      # optional region override:
      api_base: "https://bedrock-mantle.us-east-1.api.aws/v1"

  - model_name: gpt-oss-20b
    litellm_params:
      model: bedrock_mantle/openai.gpt-oss-20b
      api_key: os.environ/BEDROCK_MANTLE_API_KEY

2. 啟動 proxy

litellm --config /path/to/config.yaml

3. 傳送請求

import openai

client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000",
)

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "hello from litellm"}],
)
print(response)