跳至主要內容

Vertex AI - Anthropic、DeepSeek、Model Garden、xAI

支援的合作夥伴提供者​

提供者LiteLLM 路由Vertex 文件
Anthropic (Claude)vertex_ai/claude-*Vertex AI - Anthropic 模型
DeepSeekvertex_ai/deepseek-ai/{MODEL}Vertex AI - DeepSeek 模型
ZAI (GLM)vertex_ai/zai-org/{MODEL}Vertex AI - GLM 模型
Meta/Llamavertex_ai/meta/{MODEL}Vertex AI - Meta 模型
Mistralvertex_ai/mistral-*Vertex AI - Mistral 模型
AI21 (Jamba)vertex_ai/jamba-*Vertex AI - AI21 模型
Qwenvertex_ai/qwen/*Vertex AI - Qwen 模型
OpenAI (GPT-OSS)vertex_ai/openai/gpt-oss-*Vertex AI - GPT-OSS 模型
xAI (Grok)vertex_ai/xai/{MODEL}xAI 模型（含 Vertex）, Vertex AI Model Garden

Vertex AI - Anthropic (Claude)​

模型名稱函式呼叫
claude-3-opus@20240229completion('vertex_ai/claude-3-opus@20240229', messages)
claude-3-5-sonnet@20240620completion('vertex_ai/claude-3-5-sonnet@20240620', messages)
claude-3-sonnet@20240229completion('vertex_ai/claude-3-sonnet@20240229', messages)
claude-3-haiku@20240307completion('vertex_ai/claude-3-haiku@20240307', messages)
claude-3-7-sonnet@20250219completion('vertex_ai/claude-3-7-sonnet@20250219', messages)

使用方式​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "claude-3-sonnet@20240229"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    temperature=0.7,
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

使用方式 - thinking / reasoning_content​

from litellm import completion

resp = completion(
    model="vertex_ai/claude-3-7-sonnet-20250219",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    thinking={"type": "enabled", "budget_tokens": 1024},
)

預期回應

ModelResponse(
    id='chatcmpl-c542d76d-f675-4e87-8e5f-05855f5d0f5e',
    created=1740470510,
    model='claude-3-7-sonnet-20250219',
    object='chat.completion',
    system_fingerprint=None,
    choices=[
        Choices(
            finish_reason='stop',
            index=0,
            message=Message(
                content="The capital of France is Paris.",
                role='assistant',
                tool_calls=None,
                function_call=None,
                provider_specific_fields={
                    'citations': None,
                    'thinking_blocks': [
                        {
                            'type': 'thinking',
                            'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                            'signature': 'EuYBCkQYAiJAy6...'
                        }
                    ]
                }
            ),
            thinking_blocks=[
                {
                    'type': 'thinking',
                    'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                    'signature': 'EuYBCkQYAiJAy6AGB...'
                }
            ],
            reasoning_content='The capital of France is Paris. This is a very straightforward factual question.'
        )
    ],
    usage=Usage(
        completion_tokens=68,
        prompt_tokens=42,
        total_tokens=110,
        completion_tokens_details=None,
        prompt_tokens_details=PromptTokensDetailsWrapper(
            audio_tokens=None,
            cached_tokens=0,
            text_tokens=None,
            image_tokens=None
        ),
        cache_creation_input_tokens=0,
        cache_read_input_tokens=0
    )
)

VertexAI DeepSeek​

屬性詳細資料
提供者路由vertex_ai/deepseek-ai/{MODEL}
Vertex 文件Vertex AI - DeepSeek 模型

使用方式​

LiteLLM 支援所有 Vertex AI DeepSeek 模型。 請確保您對所有 Vertex AI DeepSeek 模型使用 vertex_ai/deepseek-ai/ 前綴。

模型名稱使用方式
vertex_ai/deepseek-ai/deepseek-r1-0528-maascompletion('vertex_ai/deepseek-ai/deepseek-r1-0528-maas', messages)

VertexAI ZAI (GLM)​

屬性詳細資料
提供者路由vertex_ai/zai-org/{MODEL}
Vertex 文件Vertex AI - GLM 模型

LiteLLM 支援所有 Vertex AI GLM 模型。 請確保您對所有 Vertex AI GLM 模型使用 vertex_ai/zai-org/ 前綴。

模型名稱使用方式
vertex_ai/zai-org/glm-4.7-maascompletion('vertex_ai/zai-org/glm-4.7-maas', messages)

使用方式​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

response = completion(
    model="vertex_ai/zai-org/glm-4.7-maas",
    messages=[{"role": "user", "content": "hi"}],
    vertex_project="your-vertex-project",
    # vertex_location routes to "global"
)
print("\nModel Response", response)

VertexAI Meta/Llama API​

模型名稱函式呼叫
meta/llama-3.2-90b-vision-instruct-maascompletion('vertex_ai/meta/llama-3.2-90b-vision-instruct-maas', messages)
meta/llama3-8b-instruct-maascompletion('vertex_ai/meta/llama3-8b-instruct-maas', messages)
meta/llama3-70b-instruct-maascompletion('vertex_ai/meta/llama3-70b-instruct-maas', messages)
meta/llama3-405b-instruct-maascompletion('vertex_ai/meta/llama3-405b-instruct-maas', messages)
meta/llama-4-scout-17b-16e-instruct-maascompletion('vertex_ai/meta/llama-4-scout-17b-16e-instruct-maas', messages)
meta/llama-4-scout-17-128e-instruct-maascompletion('vertex_ai/meta/llama-4-scout-128b-16e-instruct-maas', messages)
meta/llama-4-maverick-17b-128e-instruct-maascompletion('vertex_ai/meta/llama-4-maverick-17b-128e-instruct-maas',messages)
meta/llama-4-maverick-17b-16e-instruct-maascompletion('vertex_ai/meta/llama-4-maverick-17b-16e-instruct-maas',messages)

使用方式​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "meta/llama3-405b-instruct-maas"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI Mistral API​

支援的 OpenAI 參數

LiteLLM 支援所有 Vertex AI Mistral 模型。 請確保您對所有 Vertex AI Mistral 模型使用 vertex_ai/mistral- 前綴。

概觀

屬性詳細資料
提供者路由vertex_ai/mistral-{MODEL}
Vertex 文件Vertex AI - Mistral 模型
模型名稱函式呼叫
mistral-large@latestcompletion('vertex_ai/mistral-large@latest', messages)
mistral-large@2407completion('vertex_ai/mistral-large@2407', messages)
mistral-small-2503completion('vertex_ai/mistral-small-2503', messages)
mistral-large-2411completion('vertex_ai/mistral-large-2411', messages)
mistral-nemo@latestcompletion('vertex_ai/mistral-nemo@latest', messages)
codestral@latestcompletion('vertex_ai/codestral@latest', messages)
codestral@@2405completion('vertex_ai/codestral@2405', messages)

使用方式​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "mistral-large@2407"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

使用方式 - Codestral FIM​

透過 OpenAI /v1/completion 端點在 VertexAI 上呼叫 Codestral 以執行 FIM 任務。

注意：您也可以透過 /chat/completion 呼叫 Codestral。

from litellm import completion
import os

# os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""
# OR run `!gcloud auth print-access-token` in your terminal

model = "codestral@2405"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = text_completion(
    model="vertex_ai/" + model,
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
    prompt="def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():", 
    suffix="return True",                                              # optional
    temperature=0,                                                     # optional
    top_p=1,                                                           # optional
    max_tokens=10,                                                     # optional
    min_tokens=10,                                                     # optional
    seed=10,                                                           # optional
    stop=["return"],                                                   # optional
)

print("\nModel Response", response)

VertexAI AI21 Models​

模型名稱函式呼叫
jamba-1.5-mini@001completion(model='vertex_ai/jamba-1.5-mini@001', messages)
jamba-1.5-large@001completion(model='vertex_ai/jamba-1.5-large@001', messages)

使用方式​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "meta/jamba-1.5-mini@001"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI Qwen API​

屬性詳細資料
提供者路由vertex_ai/qwen/{MODEL}
Vertex 文件Vertex AI - Qwen 模型

LiteLLM 支援所有 Vertex AI Qwen 模型。 請確保您對所有 Vertex AI Qwen 模型使用 vertex_ai/qwen/ 前綴。

模型名稱使用方式
vertex_ai/qwen/qwen3-coder-480b-a35b-instruct-maascompletion('vertex_ai/qwen/qwen3-coder-480b-a35b-instruct-maas', messages)
vertex_ai/qwen/qwen3-235b-a22b-instruct-2507-maascompletion('vertex_ai/qwen/qwen3-235b-a22b-instruct-2507-maas', messages)

使用方式​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "qwen/qwen3-coder-480b-a35b-instruct-maas"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI GPT-OSS Models​

屬性詳細資料
提供者路由vertex_ai/openai/{MODEL}
Vertex 文件Vertex AI - GPT-OSS 模型

LiteLLM 支援所有 Vertex AI GPT-OSS 模型。 請確保您對所有 Vertex AI GPT-OSS 模型使用 vertex_ai/openai/ 前綴。

模型名稱用量
vertex_ai/openai/gpt-oss-20b-maascompletion('vertex_ai/openai/gpt-oss-20b-maas', messages)

用量​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "openai/gpt-oss-20b-maas"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

用量 - reasoning_effort​

GPT-OSS 模型支援 reasoning_effort 參數，以提供增強的推理能力。

from litellm import completion

response = completion(
    model="vertex_ai/openai/gpt-oss-20b-maas",
    messages=[{"role": "user", "content": "Solve this complex problem step by step"}],
    reasoning_effort="low",  # Options: "minimal", "low", "medium", "high"
    vertex_ai_project="your-vertex-project",
    vertex_ai_location="us-central1",
)

VertexAI xAI (Grok)​

Vertex AI Model Garden 中可用的 xAI Grok 模型，使用與其他 Model Garden 發佈者模型相同的 OpenAI 相容 chat-completions 路徑。請使用 vertex_ai/xai/ 前綴（不是 xai/，後者是帶有 XAI_API_KEY 的直接 xAI API）。另請參閱 xAI provider。

屬性詳細資料
提供者路由vertex_ai/xai/{MODEL}
Vertex / xAI 文件xAI models, Model Garden
模型名稱用量
vertex_ai/xai/grok-4.1-fast-non-reasoningcompletion('vertex_ai/xai/grok-4.1-fast-non-reasoning', messages)
vertex_ai/xai/grok-4.1-fast-reasoningcompletion('vertex_ai/xai/grok-4.1-fast-reasoning', messages)
vertex_ai/xai/grok-4.20-non-reasoningcompletion('vertex_ai/xai/grok-4.20-non-reasoning', messages)
vertex_ai/xai/grok-4.20-reasoningcompletion('vertex_ai/xai/grok-4.20-reasoning', messages)

用量​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

vertex_ai_project = "your-vertex-project"  # or os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location"  # or os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/xai/grok-4.1-fast-non-reasoning",
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)
🚅
LiteLLM Enterprise
為正式環境打造的 SSO/SAML、稽核記錄、支出追蹤、多團隊管理與防護欄。
深入瞭解 →