[Proposal] Add AI Provider Request Context to AI Gateway

[pamaljayasinghe]

New AI Gateway Request Context for AI Providers

Summary

Introduce a new AI-specific request context for AI provider traffic. The gateway should expose provider and model information through a structured context such as:

request.ai.provider
request.ai.model

This context will be available during request processing and in request update events so policies, analytics, logging, and other gateway features can consistently identify which AI provider and model are being used.

---

Motivation

AI gateway traffic needs provider-aware and model-aware metadata. Today, policies and request update flows can depend on provider or model information, but that information is not represented as a consistent AI request context.

This request context can be used to work across multiple providers and can support future routing decisions based on the selected provider and model. Adding request.ai.provider and request.ai.model gives the gateway a single, predictable place to read AI request metadata.

---

Proposal

Add a new AI request context namespace called request.ai.

The gateway should populate the following fields for AI provider requests:

Context field	Description
request.ai.provider	The resolved AI provider identifier used by the request.
request.ai.model	The resolved AI model identifier used by the request.

The provider value should come from the matched AI provider configuration.

The model value should come from the configured request model extraction logic for that provider template, such as:

• Payload extraction
• Header extraction
• Query parameter extraction
• Path parameter extraction

Important compatibility note

• This feature only works with Gateway 1.2.0 and later.
• Older request update formats are not expected to support this feature.
• Consumers that continue to use old request context fields may break or miss provider and model values.
• Existing consumers must be updated to read:
  • request.ai.provider
  • request.ai.model

---

Example Flow

The following example demonstrates how AI provider and model context can be shared between policies.

[1] Intelligent Model Routing Policy

This is an example routing policy responsible for making intelligent routing decisions.

When the policy selects a model/provider combination, it stores the resolved values into the request context:

request.ai.provider
request.ai.model

This allows downstream policies to consistently consume the selected AI routing metadata.

[2] OpenAI to Anthropic AI Provider Changing Policy

This is a provider transformation policy currently under development.

The policy reads:

request.ai.provider
request.ai.model

from the request context and performs provider transformation logic based on those values.

For example, if the routing policy originally selected:

provider = openai
model = gpt-4o

the transformation policy may decide to reroute the request to:

provider = anthropic
model = claude-sonnet-4

before forwarding the request to the final upstream provider endpoint.

---

[pamaljayasinghe]

Update – Currently, this process has been stopped because other policies, such as rate-limiting, use shared context metadata. Also, implementing this with the shared context metadata policy will allow it to work with any gateway version(new or old).