Passing headers with the Azure OpenAI client in 1.1.1

[dkhundley]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

To be clear, I’m not certain this is a bug, but I’m uncertain how to do this now.

For extra security layers, I am exposing Azure OpenAI via the Azure API Management service. When I make a call this way, I pass a bearer JWT token and Apim subscription key as headers. I had no problem doing this on lower versions like 0.28.1, but now I don’t know how to do this.

To Reproduce

Attempting to call Azure OpenAI with 1.1.1. The call does not successfully connect. After glancing through the source Python code, I thought maybe sticking the headers as default_headers in the AzureOpenAI client would work, but it does not.

# Setting the auth headers
auth_headers = {
    ‘Ocp-Apim-Subscription-Key’: my_apim_sub_key,
    ‘Authorization’: f’Bearer {my_jwt_token}’
}

# Setting the Azure OpenAI client
azure_openai_client = AzureOpenAI(
   api_key = ‘null’,
   api_version = ‘2023-07-01-preview’,
   azure_endpoint = my_special_url,
   default_headers = auth_headers
)

# Calling Azure OpenAI
openai_response = azure_openai_client.chat.completions.create(
    model = ‘gpt-35-turbo-16k’,
    messages = my_messages
)

Here’s the way I’m doing it on 0.28.1, and this still works just fine. As you can see, I’m passing the auth headers at the time of the actual API call here.

import openai

# Setting the auth headers
auth_headers = {
    ‘Ocp-Apim-Subscription-Key’: my_apim_sub_key,
    ‘Authorization’: f’Bearer {my_jwt_token}’
}

# Setting config information on OpenAI
openai.api_url = my_special_url
openai.api_version = ‘2023-07-01-preview’
openai.api_key = ‘null’

# Calling Azure OpenAI
openai_response = openai.ChatCompletion.create(
    engine = ‘gpt-35-turbo-16k’,
    messages = my_messages,
    Headers = auth_headers
)

What might I be doing wrong?

Code snippets

No response

OS

macOS

Python version

Python v3.10

Library version

1.1.1

[RobertCraigie]

Hey @dkhundley, I can't reproduce the issue you're seeing.

I've updated your snippet to pass in an event hook to easily verify what headers are being sent in the request:

import json
import httpx
from openai import AzureOpenAI

my_apim_sub_key = "apim key"
my_jwt_token = "jwt token"
my_special_url = "https://unknown_url.azure.com"
auth_headers = {
    "Ocp-Apim-Subscription-Key": my_apim_sub_key,
    "Authorization": f"Bearer {my_jwt_token}",
}

def event_hook(req: httpx.Request) -> None:
    print(json.dumps(dict(req.headers), indent=2))
    raise BaseException()

azure_openai_client = AzureOpenAI(
    api_key="null",
    api_version="2023-07-01-preview",
    azure_endpoint=my_special_url,
    default_headers=auth_headers,
    http_client=httpx.Client(event_hooks={"request": [event_hook]}),
)
openai_response = azure_openai_client.chat.completions.create(
    model="gpt-35-turbo-16k",
    messages=[],
)

This is the log I get:

{
  "host": "unknown_url.azure.com",
  "accept-encoding": "gzip, deflate",
  "connection": "keep-alive",
  "accept": "application/json",
  "content-type": "application/json",
  "user-agent": "AzureOpenAI/Python 0.0.1",
  "x-stainless-lang": "python",
  "x-stainless-package-version": "0.0.1",
  "x-stainless-os": "MacOS",
  "x-stainless-arch": "arm64",
  "x-stainless-runtime": "CPython",
  "x-stainless-runtime-version": "3.9.18",
  "authorization": "Bearer jwt token",
  "ocp-apim-subscription-key": "apim key",
  "api-key": "null",
  "content-length": "45"
}

Which seems to match what you want? The authorization and ocp-apim-subscription-key headers are present.

[dkhundley]

Hmmm okay. I'm away from my work computer for the remainder of the day, but I'll see what I can try and post the precise error that I'm seeing. Thank you for the quick reply!

[dkhundley]

@RobertCraigie Well, I don't know if I was just making a silly mistake or if something was fixed in 1.1.2, but I just updated my client to 1.1.2 and now I'm totally fine. Thank you again for looking into this!