/rerank

提示

LiteLLM 遵循 rerank API 的 cohere API 請求 / 回應

概觀

功能	支援	備註
成本追蹤	✅	適用於所有支援的模型
記錄	✅	適用於所有整合
終端使用者追蹤	✅
備援	✅	可在支援的模型之間運作
負載平衡	✅	可在支援的模型之間運作
防護欄	✅	僅套用於輸入查詢（不包含文件）
支援的提供者	Cohere, Together AI, Azure AI, DeepInfra, Nvidia NIM, Infinity, Fireworks AI, Voyage AI, watsonx.ai

LiteLLM Python SDK 用法

快速開始

from litellm import rerank
import os

os.environ["COHERE_API_KEY"] = "sk-.."

query = "What is the capital of the United States?"
documents = [
    "Carson City is the capital city of the American state of Nevada.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
    "Washington, D.C. is the capital of the United States.",
    "Capital punishment has existed in the United States since before it was a country.",
]

response = rerank(
    model="cohere/rerank-english-v3.0",
    query=query,
    documents=documents,
    top_n=3,
)
print(response)

非同步用法

from litellm import arerank
import os, asyncio

os.environ["COHERE_API_KEY"] = "sk-.."

async def test_async_rerank(): 
    query = "What is the capital of the United States?"
    documents = [
        "Carson City is the capital city of the American state of Nevada.",
        "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
        "Washington, D.C. is the capital of the United States.",
        "Capital punishment has existed in the United States since before it was a country.",
    ]

    response = await arerank(
        model="cohere/rerank-english-v3.0",
        query=query,
        documents=documents,
        top_n=3,
    )
    print(response)

asyncio.run(test_async_rerank())

LiteLLM Proxy 用法

LiteLLM 提供與 cohere API 相容的 /rerank 端點供 Rerank 呼叫使用。

設定

將以下內容新增至您的 litellm proxy config.yaml

model_list:
  - model_name: Salesforce/Llama-Rank-V1
    litellm_params:
      model: together_ai/Salesforce/Llama-Rank-V1
      api_key: os.environ/TOGETHERAI_API_KEY
  - model_name: rerank-english-v3.0
    litellm_params:
      model: cohere/rerank-english-v3.0
      api_key: os.environ/COHERE_API_KEY

啟動 litellm

litellm --config /path/to/config.yaml

# RUNNING on http://0.0.0.0:4000

測試請求

curl http://0.0.0.0:4000/rerank \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rerank-english-v3.0",
    "query": "What is the capital of the United States?",
    "documents": [
        "Carson City is the capital city of the American state of Nevada.",
        "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
        "Washington, D.C. is the capital of the United States.",
        "Capital punishment has existed in the United States since before it was a country."
    ],
    "top_n": 3
  }'

支援的提供者

⚡️請參閱 models.litellm.ai 上所有支援的模型與提供者

提供者	用法連結
Cohere (v1 + v2 clients)	用法
Together AI	用法
Azure AI	用法
Jina AI	用法
AWS Bedrock	用法
HuggingFace	用法
Infinity	用法
vLLM	用法
DeepInfra	用法
Vertex AI	用法
Fireworks AI	用法
Voyage AI	用法
IBM watsonx.ai	用法