fix: normalize model names, fix claude 4.8

[k11kirky]

Problem

LiteLLM uses provider-prefixed model names (e.g. anthropic/claude-opus-4-8) for routing. When bare model IDs were passed to the Anthropic endpoint, LiteLLM had to infer the provider, which could lead to ambiguous or incorrect routing. The OpenAI endpoint already had a local helper to handle this, but it was narrow in scope and not reusable.

Changes

Introduces a shared normalize_litellm_model_name(model, provider) utility in handler.py that prepends an explicit provider prefix to any model name that doesn't already carry a known LiteLLM provider prefix. The Anthropic endpoint now uses this to ensure model names are prefixed before being forwarded to LiteLLM. The existing OpenAI-specific _normalize_model_name helper is removed and replaced with calls to the shared utility.

How did you test this code?

A new test (test_bare_claude_model_is_prefixed_for_litellm_routing) verifies that a bare model ID sent to the Anthropic /v1/messages endpoint is forwarded to LiteLLM as anthropic/<model>. Existing tests were updated to assert the prefixed model name, and the 429-fallback test now additionally asserts that the Bedrock retry uses the correct bedrock/ prefixed model name.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

[k11kirky]

• fix: normalize model names, fix claude 4.8 #60631 👈 (View in Graphite)
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Comments Outside Diff (1)

1. services/llm-gateway/src/llm_gateway/api/anthropic.py, line 215-222 (link)

   model label in metrics differs from the model sent to LiteLLM

   handle_llm_request receives model=body.model (bare, e.g. claude-opus-4-8) while request_data["model"] is the normalized anthropic/claude-opus-4-8. This bare value is used for all Prometheus labels and structured-log bindings, so any Anthropic request will surface under the unprefixed model name in dashboards while the sibling _handle_responses handler emits openai/<model> in its metrics. The _handle_responses handler passes model=normalized_model to both data["model"] and handle_llm_request, so that endpoint is self-consistent. Passing model=body.model instead of the normalized model is harmless at runtime, but creates a label inconsistency across endpoints.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: services/llm-gateway/src/llm_gateway/api/anthropic.py
   Line: 215-222
   
   Comment:
   **`model` label in metrics differs from the model sent to LiteLLM**
   
   `handle_llm_request` receives `model=body.model` (bare, e.g. `claude-opus-4-8`) while `request_data["model"]` is the normalized `anthropic/claude-opus-4-8`. This bare value is used for all Prometheus labels and structured-log bindings, so any Anthropic request will surface under the unprefixed model name in dashboards while the sibling `_handle_responses` handler emits `openai/<model>` in its metrics. The `_handle_responses` handler passes `model=normalized_model` to both `data["model"]` and `handle_llm_request`, so that endpoint is self-consistent. Passing `model=body.model` instead of the normalized model is harmless at runtime, but creates a label inconsistency across endpoints.
   
   How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
services/llm-gateway/src/llm_gateway/api/anthropic.py:215-222
**`model` label in metrics differs from the model sent to LiteLLM**

`handle_llm_request` receives `model=body.model` (bare, e.g. `claude-opus-4-8`) while `request_data["model"]` is the normalized `anthropic/claude-opus-4-8`. This bare value is used for all Prometheus labels and structured-log bindings, so any Anthropic request will surface under the unprefixed model name in dashboards while the sibling `_handle_responses` handler emits `openai/<model>` in its metrics. The `_handle_responses` handler passes `model=normalized_model` to both `data["model"]` and `handle_llm_request`, so that endpoint is self-consistent. Passing `model=body.model` instead of the normalized model is harmless at runtime, but creates a label inconsistency across endpoints.

_{Reviews (1): Last reviewed commit: "fix: normalize model names, fix claude 4..." | Re-trigger Greptile}

[joshsny]

LGTM, assuming it's tested locally

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-05-29 10:38 UTC	Run
prod-us	✅ Deployed	2026-05-29 10:51 UTC	Run
prod-eu	✅ Deployed	2026-05-29 11:01 UTC	Run