[BUG] kagent does not use api key when using api key auth enabled Agentgateway

[Conor-Behard333]

📋 Prerequisites

• I have searched the existing issues to avoid creating a duplicate
• By submitting this issue, you agree to follow our Code of Conduct
• I am using the latest version of the software
• I have tried to clear cache/cookies or used incognito mode (if ui-related)
• I can consistently reproduce this issue

🎯 Affected Service(s)

Not Sure

🚦 Impact/Severity

Blocker

🐛 Bug Description

I have deployed agentgateway to act as a proxy between azure openai and kagent.

when there is no virtual key policies on the agent gateway it is able to talk to the azure openAI without any issues.

Once I add a agentgateway policy to force an api key to access the llm via the gateway kagent is unable to find the api key and use it in the request to the gateway.

🔄 Steps To Reproduce

1. Deploy agentgateway
2. Connect agentgateway to azure openai (https://agentgateway.dev/docs/kubernetes/latest/setup/gateway/)
3. Add an api key via agentgateway policy (https://agentgateway.dev/docs/kubernetes/latest/llm/virtual-keys/)
4. configure the model in kagent to use the agentgateway gateway

{
  "azureEndpoint": "https://agentgateway.local/",
  "apiVersion": "v1",
  "azureDeployment": "gpt-4.1-mini"
}

5. add the api key from the agentgateway policy to the model config
6. send a prompt to the llm via an agent using the model config

🤔 Expected Behavior

prompt should pass through the gateway and you should receive a response from the llm

📱 Actual Behavior

Agent responds with "api key authentication failure: no API Key found."

💻 Environment

• kubernetes version: 1.35.0
• provider: azure
• app version: v0.9.11

🔧 CLI Bug Report

POST https://DOMAIN/openai/deployments/gpt-4.1-mini/chat/completions?api-version=v1 "HTTP/1.1 401 Unauthorized"

🔍 Additional Context

• If I remove the auth policy on the agentgateway backend it will begin to work again
• We have used other applications like gitlab duo and we have not had the same issue which indicates it is specifically an issue with kagent
• I have shelled into the agent pod and ran:

env | grep AZURE

and the output of the base url and the api key are correct.

📋 Logs

📷 Screenshots

No response

🙋 Are you willing to contribute?

• I am willing to submit a PR to fix this issue

[markseibal-png]

The API key error points to a provider-side failure, not a local code bug.

npx ai-doctor@latest --json

Scans your API config for timeout, retry, and error-handling gaps in one pass.

[Conor-Behard333]

where should i be running this and how do I configure it to reference our gateway fqdn?

[Conor-Behard333]

for more information we deploy the model via the Kagent helm chart:

values:
    providers:
      default: azureOpenAI
      azureOpenAI:
        provider: AzureOpenAI
        model: "gpt-4.1-mini"
        apiKeySecretRef: kagent-gateway-key
        apiKeySecretKey: kagent-gateway-key
        tls:
          disableVerify: false
          caCertSecretRef: "ca-cert"
          caCertSecretKey: "ca-cert"
          disableSystemCAs: false
        config:
          apiVersion: "v1"
          azureDeployment: "gpt-4.1-mini"
          azureEndpoint: "https://<agent-gateway-fqdn>/"

As stated before in the initial description it works fine when we remove the requirement for api keys on the agentgateway. but as soon as we enable them we get the "api key authentication failure: no API Key found" even tho the api key is clearly set in the agent pod