Azure OpenAI prompt caching broken for gpt-5.2 model (works directly on Azure)

[AdithyanI]

Summary

Azure OpenAI supports prompt caching via the prompt_cache_key parameter. This allows you to cache the prompt prefix so that repeated calls with the same prefix are cheaper and faster.

The problem: When using LiteLLM proxy, prompt caching works for gpt-5.1 but completely fails for gpt-5.2. However, when I call Azure OpenAI directly (bypassing LiteLLM), prompt caching works for BOTH models.

This means LiteLLM is doing something that breaks prompt caching specifically for gpt-5.2.

---

How I Tested This

I ran the same test twice:

1. Through LiteLLM proxy - calling my LiteLLM instance
2. Direct to Azure OpenAI - calling Azure directly, bypassing LiteLLM

For each test, I made 5 API calls with the same prompt (~2000 tokens) and the same prompt_cache_key. After the first call, subsequent calls should show cached_tokens > 0 in the response.

---

Results

Test 1: Through LiteLLM Proxy

Model	Cache Hits
gpt-5.1	5/5 (100%) ✅
gpt-5.2	0/5 (0%) ❌

Test 2: Direct to Azure OpenAI (no LiteLLM)

Model	Cache Hits
gpt-5.1	5/5 (100%) ✅
gpt-5.2	4/5 (80%) ✅

Conclusion: gpt-5.2 DOES support prompt caching on Azure. The issue is that LiteLLM is breaking it somehow.

---

Steps to Reproduce

1. LiteLLM Config

Both models are configured identically in my config.yaml:

- litellm_params:
    api_base: https://my-resource.openai.azure.com/
    api_key: <key>
    model: azure/gpt-5.1
    api_version: "2024-08-01-preview"
    drop_params: false
  model_name: gpt-5.1

- litellm_params:
    api_base: https://my-resource.openai.azure.com/
    api_key: <key>
    model: azure/gpt-5.2
    api_version: "2024-08-01-preview"
    drop_params: false
  model_name: gpt-5.2

2. Test Code (through LiteLLM)

from openai import OpenAI

client = OpenAI(
    base_url="https://my-litellm-proxy.com/",
    api_key="sk-xxx",
)

# Create a prompt with >1024 tokens (required for caching)
LONG_PROMPT = "You are an expert. " + ("Test content. " * 200)

# Make 5 calls with the same cache key
for i in range(5):
    resp = client.chat.completions.create(
        model="gpt-5.2",  # Change to gpt-5.1 to see it work
        messages=[{"role": "user", "content": LONG_PROMPT}],
        max_tokens=100,
        extra_body={"prompt_cache_key": "my_cache_key"},
    )
    
    cached = resp.usage.prompt_tokens_details.cached_tokens
    total = resp.usage.prompt_tokens
    print(f"Call {i+1}: cached={cached}/{total}")

# Expected: Call 2-5 should show cached > 0
# Actual with gpt-5.2: ALL calls show cached=0
# Actual with gpt-5.1: Calls 2-5 show cached=1920/1996 (96%)

3. Test Code (Direct Azure - to prove caching works)

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://my-resource.openai.azure.com/",
    api_key="<key>",
    api_version="2024-08-01-preview",
)

LONG_PROMPT = "You are an expert. " + ("Test content. " * 200)

for i in range(5):
    resp = client.chat.completions.create(
        model="gpt-5.2",
        messages=[{"role": "user", "content": LONG_PROMPT}],
        max_completion_tokens=100,
        extra_body={"prompt_cache_key": "my_cache_key"},
    )
    
    cached = resp.usage.prompt_tokens_details.cached_tokens
    total = resp.usage.prompt_tokens
    print(f"Call {i+1}: cached={cached}/{total}")

# Result: Calls 2-5 show cached=1792/1996 (90%) - IT WORKS!

---

Environment

• LiteLLM Version: v1.80.8-stable
• Deployment: LiteLLM Proxy (Docker)
• Python: 3.12
• Azure OpenAI API Version: 2024-08-01-preview
• Models: gpt-5.1 (2025-11-13) and gpt-5.2 (2025-12-11)

---

Expected Behavior

When I pass prompt_cache_key in extra_body, LiteLLM should forward this parameter to Azure OpenAI for both gpt-5.1 and gpt-5.2. Both models should show prompt caching working.

---

Actual Behavior

• gpt-5.1: Prompt caching works ✅
• gpt-5.2: Prompt caching completely broken (0% cache hits) ❌

---

What I've Already Tried

1. Set drop_params: false in config for both models
2. Verified both models use identical config
3. Tested with fresh cache keys each time
4. Confirmed gpt-5.2 caching works when calling Azure directly

---

My Hypothesis

LiteLLM may have model-specific handling that's different for gpt-5.2 vs gpt-5.1. Possibly:

• The prompt_cache_key parameter is being dropped/filtered for gpt-5.2
• There's a code path that modifies the request differently for newer model versions
• Some parameter transformation is breaking the caching mechanism

---

Workaround

For now, I'm using gpt-5.1 when I need prompt caching.