---
title: "Custom Prompt Management"
url: "/en/docs/proxy/custom_prompt_management"
canonical_url: "https://docs.litellm.ai/en/docs/proxy/custom_prompt_management"
type: "docs"
summary: "Connect LiteLLM to your prompt management system with custom hooks."
related:
  - "/en/docs/proxy/litellm_prompt_management"
  - "/en/docs/proxy/native_litellm_prompt"
---
# Custom Prompt Management

> Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt


Connect LiteLLM to your prompt management system with custom hooks.

## Overview

## How it works

## Quick Start

### 1. Create Your Custom Prompt Manager

Create a class that inherits from `CustomPromptManagement` to handle prompt retrieval and formatting:

**Example Implementation**

Create a new file called `custom_prompt.py` and add this code. The key method here is `get_chat_completion_prompt` you can implement custom logic to retrieve and format prompts based on the `prompt_id` and `prompt_variables`.

```python
from typing import List, Tuple, Optional
from litellm.integrations.custom_prompt_management import CustomPromptManagement
from litellm.types.llms.openai import AllMessageValues
from litellm.types.prompts.init_prompts import PromptSpec
from litellm.types.utils import StandardCallbackDynamicParams

class MyCustomPromptManagement(CustomPromptManagement):
    def get_chat_completion_prompt(
        self,
        model: str,
        messages: List[AllMessageValues],
        non_default_params: dict,
        prompt_id: str,
        prompt_variables: Optional[dict],
        dynamic_callback_params: StandardCallbackDynamicParams,
        prompt_spec: Optional[PromptSpec] = None,
        prompt_label: Optional[str] = None,
        prompt_version: Optional[int] = None,
        ignore_prompt_manager_model: Optional[bool] = False,
        ignore_prompt_manager_optional_params: Optional[bool] = False,
    ) -> Tuple[str, List[AllMessageValues], dict]:
        """
        Retrieve and format prompts based on prompt_id.
        
        Returns:
            - model: The model to use
            - messages: The formatted messages
            - non_default_params: Optional parameters like temperature
        """
        # Example matching the diagram: Add system message for prompt_id "1234"
        if prompt_id == "1234":
            # Prepend system message while preserving existing messages
            new_messages = [
                {"role": "system", "content": "Be a good Bot!"},
            ] + messages
            return model, new_messages, non_default_params
        
        # Default: Return original messages if no prompt_id match
        return model, messages, non_default_params

prompt_management = MyCustomPromptManagement()
```

### 2. Configure Your Prompt Manager in LiteLLM `config.yaml`

```yaml
model_list:
  - model_name: gpt-5.6-terra
    litellm_params:
      model: openai/gpt-5.6-terra
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  callbacks: custom_prompt.prompt_management  # sets litellm.callbacks = [prompt_management]
```

### 3. Start LiteLLM Gateway

**Docker Run**

Mount your `custom_logger.py` on the LiteLLM Docker container.

```shell
docker run -d \
  -p 4000:4000 \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  --name my-app \
  -v $(pwd)/my_config.yaml:/app/config.yaml \
  -v $(pwd)/custom_logger.py:/app/custom_logger.py \
  my-app:latest \
  --config /app/config.yaml \
  --port 4000 \
  --detailed_debug \
```

**litellm pip**

```shell
litellm --config config.yaml --detailed_debug
```

### 4. Test Your Custom Prompt Manager

When you pass `prompt_id="1234"`, the custom prompt manager will add a system message "Be a good Bot!" to your conversation:

**OpenAI Python v1.0.0+**

```python
from openai import OpenAI

client = OpenAI(
    api_key="sk-<your-litellm-api-key>",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="gpt-5.6-terra",
    messages=[{"role": "user", "content": "hi"}],
    extra_body={
        "prompt_id": "1234"
    }
)

print(response.choices[0].message.content)
```

**Langchain**

```python
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage

chat = ChatOpenAI(
    model="gpt-5.6-terra",
    openai_api_key="sk-<your-api-key>",
    openai_api_base="http://0.0.0.0:4000",
    extra_body={
        "prompt_id": "1234"
    }
)

messages = []
response = chat(messages)

print(response.content)
```

**Curl**

```shell
curl -X POST http://0.0.0.0:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d '{
    "model": "gpt-5.6-terra",
    "messages": [{"role": "user", "content": "hi"}],
    "prompt_id": "1234"
}'
```

### Using the LiteLLM SDK Directly

If you call `litellm.completion()` from a Python script (without going through the proxy), register your custom prompt manager before making the request:

```python

import litellm
from custom_prompt import prompt_management

litellm.callbacks = [prompt_management]

response = litellm.completion(
    model="gpt-5.6-terra",
    messages=[{"role": "user", "content": "hi"}],
    prompt_id="1234",
    prompt_variables={"user_message": "hi"},
)
```

> **Note:** `litellm.callbacks = [prompt_management]` (or equivalently `litellm.logging_callback_manager.add_litellm_callback(prompt_management)`) is required in SDK scripts. The proxy reads `callbacks` from `config.yaml` automatically, but standalone scripts do not.

The request will be transformed from:
```json
{
    "model": "gpt-5.6-terra",
    "messages": [{"role": "user", "content": "hi"}],
    "prompt_id": "1234"
}
```

To:
```json
{
    "model": "gpt-5.6-terra",
    "messages": [
        {"role": "system", "content": "Be a good Bot!"},
        {"role": "user", "content": "hi"}
    ]
}
```

## Related pages

- [LiteLLM AI Gateway Prompt Management](https://docs.litellm.ai/en/docs/proxy/litellm_prompt_management.md)
- [LiteLLM Prompt Management (GitOps)](https://docs.litellm.ai/en/docs/proxy/native_litellm_prompt.md)