跳至主要內容

Cohere

API 金鑰​

import os 
os.environ["COHERE_API_KEY"] = ""

使用方式​

LiteLLM Python SDK​

Cohere v2 API（預設）​

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v2 call
response = completion(
    model="cohere_chat/command-a-03-2025", 
    messages = [{ "content": "Hello, how are you?","role": "user"}]
)

Cohere v1 API​

若要使用 Cohere v1/chat API，請在模型名稱前加上 cohere_chat/v1/：

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v1 call
response = completion(
    model="cohere_chat/v1/command-a-03-2025", 
    messages = [{ "content": "Hello, how are you?","role": "user"}]
)

串流​

Cohere v2 串流：

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v2 streaming
response = completion(
    model="cohere_chat/command-a-03-2025", 
    messages = [{ "content": "Hello, how are you?","role": "user"}],
    stream=True
)

for chunk in response:
    print(chunk)

Cohere v1 串流：

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v1 streaming
response = completion(
    model="cohere_chat/v1/command-a-03-2025", 
    messages = [{ "content": "Hello, how are you?","role": "user"}],
    stream=True
)

for chunk in response:
    print(chunk)

與 LiteLLM Proxy 搭配使用​

以下是如何透過 LiteLLM Proxy Server 呼叫 Cohere

1. 將金鑰儲存在您的環境中​

export COHERE_API_KEY="your-api-key"

2. 啟動 proxy​

在 config.yaml 中定義您要使用的 cohere 模型

適用於 Cohere v1 模型：

model_list:
  - model_name: command-a-03-2025 
    litellm_params:
      model: cohere_chat/v1/command-a-03-2025
      api_key: "os.environ/COHERE_API_KEY"

適用於 Cohere v2 模型：

model_list:
  - model_name: command-a-03-2025-v2
    litellm_params:
      model: cohere_chat/command-a-03-2025
      api_key: "os.environ/COHERE_API_KEY"
litellm --config /path/to/config.yaml

3. 測試​

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <your-litellm-api-key>' \
--data ' {
      "model": "command-a-03-2025",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ]
    }
'

支援的模型​

模型名稱函式呼叫
command-a-03-2025litellm.completion('command-a-03-2025', messages)
command-r-plus-08-2024litellm.completion('command-r-plus-08-2024', messages)
command-r-08-2024litellm.completion('command-r-08-2024', messages)
command-r-pluslitellm.completion('command-r-plus', messages)
command-rlitellm.completion('command-r', messages)
command-lightlitellm.completion('command-light', messages)
command-nightlylitellm.completion('command-nightly', messages)

嵌入​

from litellm import embedding
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere call
response = embedding(
    model="embed-english-v3.0", 
    input=["good morning from litellm", "this is another item"], 
)

設定 - v3 模型的輸入類型​

v3 模型有一個必填參數：input_type。LiteLLM 預設為 search_document。它可以是以下四個值之一：

  • input_type="search_document"： （預設）當您要將文字（文件）儲存在向量資料庫中時使用
  • input_type="search_query"：當您要用搜尋查詢在向量資料庫中尋找最相關的文件時使用
  • input_type="classification"：當您將 embeddings 作為分類系統的輸入時使用
  • input_type="clustering"：當您將 embeddings 用於文字叢集時使用

https://txt.cohere.com/introducing-embed-v3/

from litellm import embedding
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere call
response = embedding(
    model="embed-english-v3.0", 
    input=["good morning from litellm", "this is another item"], 
    input_type="search_document" 
)

支援的嵌入模型​

模型名稱函式呼叫
embed-english-v3.0embedding(model="embed-english-v3.0", input=["good morning from litellm", "this is another item"])
embed-english-light-v3.0embedding(model="embed-english-light-v3.0", input=["good morning from litellm", "this is another item"])
embed-multilingual-v3.0embedding(model="embed-multilingual-v3.0", input=["good morning from litellm", "this is another item"])
embed-multilingual-light-v3.0embedding(model="embed-multilingual-light-v3.0", input=["good morning from litellm", "this is another item"])
embed-english-v2.0embedding(model="embed-english-v2.0", input=["good morning from litellm", "this is another item"])
embed-english-light-v2.0embedding(model="embed-english-light-v2.0", input=["good morning from litellm", "this is another item"])
embed-multilingual-v2.0embedding(model="embed-multilingual-v2.0", input=["good morning from litellm", "this is another item"])

重新排序​

使用方式​

LiteLLM 支援 Cohere rerank 的 v1 和 v2 用戶端。預設情況下，rerank 端點會使用 v2 用戶端，但您可以透過明確呼叫 v1/rerank 來指定 v1 用戶端

from litellm import rerank
import os

os.environ["COHERE_API_KEY"] = "sk-.."

query = "What is the capital of the United States?"
documents = [
    "Carson City is the capital city of the American state of Nevada.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
    "Washington, D.C. is the capital of the United States.",
    "Capital punishment has existed in the United States since before it was a country.",
]

response = rerank(
    model="cohere/rerank-english-v3.0",
    query=query,
    documents=documents,
    top_n=3,
)
print(response)

解析（OCR）​

Cohere Parse 會將文件影像轉換為 markdown。LiteLLM 透過 /ocr 端點 提供此功能，因此請求與回應會使用與其他 OCR 提供者相同的格式，而且每次呼叫都會依據計費頁面進行成本追蹤。

Parse 僅接受 image_url 文件：影像 URL 或 base64 data:image/... URI。PDF 與 document_url 輸入會在送往 Cohere 之前先以 400 拒絕。

import os
from litellm import ocr

os.environ["COHERE_API_KEY"] = ""

response = ocr(
    model="cohere/parse-v5.0",
    document={
        "type": "image_url",
        "image_url": "https://raw.githubusercontent.com/mistralai/cookbook/refs/heads/main/mistral/ocr/receipt.png",
    },
)

for page in response.pages:
    print(page.markdown)
print(response.usage_info.pages_processed)

支援的參數​

參數值說明
output_formatmarkdown（預設）、blocksCohere 的輸出版面配置。使用 blocks 時，每一頁都會包含 Cohere 的 blocks 陣列，且 markdown 會是空的
req_formatlitellm（預設）、nativenative 會回傳 Cohere 自己的回應主體，而不是 LiteLLM OCR 格式

部署於 Azure AI Foundry 的 Cohere Parse 已涵蓋於 Azure AI OCR。

LiteLLM Enterprise
為正式環境打造的 SSO/SAML、稽核記錄、支出追蹤、多團隊管理與防護欄。
深入瞭解 →