AzureOpenAI authentication issue: new token from azure_ad_token_provider not utilized after expiration

[maozguttman]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

When the initial token expires during a series of retries due to multiple failures, the azure_ad_token_provider generates a new token as expected. However, this new token is not utilized by AzureOpenAI for the remaining retries.

The problem originates from the openai/lib/azure.py -> _prepare_options function, where the azure_ad_token is only set to headers["Authorization"] during initialization and not when azure_ad_token is changed.
See if headers.get("Authorization") is None below:

    def _prepare_options(self, options: FinalRequestOptions) -> None:
        headers: dict[str, str | Omit] = {**options.headers} if is_given(options.headers) else {}
        options.headers = headers

        azure_ad_token = self._get_azure_ad_token()
        if azure_ad_token is not None:
            if headers.get("Authorization") is None:
                headers["Authorization"] = f"Bearer {azure_ad_token}"

To Reproduce

The azure.identity.ClientSecretCredential.get_token function was utilized as the azure_ad_token_provider, and the AzureOpenAI.max_retries was set to 1000. Due to numerous failures and retries, the process exceeded the 60-minute validity period of the token. It is important to note that Langchain is being used, rather than directly interfacing with AzureOpenAI.

Code snippets

No response

OS

SuSE12

Python version

Python 3.9.6

Library version

openai v1.27.0

[rattrayalex]

Thank you for reporting!

How were you initializing the library with langchain? Were you using the Azure version?

Regardless, this does look like a bug to me as well:

openai-python/src/openai/lib/azure.py

Lines 268 to 281 in 50371bf

	def _get_azure_ad_token(self) -> str | None:
	if self._azure_ad_token is not None:
	return self._azure_ad_token
	provider = self._azure_ad_token_provider
	if provider is not None:
	token = provider()
	if not token or not isinstance(token, str): # pyright: ignore[reportUnnecessaryIsInstance]
	raise ValueError(
	f"Expected `azure_ad_token_provider` argument to return a string but it returned {token}",
	)
	return token
	return None

cc @RobertCraigie @kristapratico

[maozguttman]

Yes, instantiating a langchain_openai.AzureChatOpenAI Langchain object and passing max_retries (which is passed to OpenAI AzureOpenAI) as following:

from langchain_openai import AzureChatOpenAI

        llm: AzureChatOpenAI = AzureChatOpenAI(
            azure_deployment=<VALUE>,
            api_version=<VALUE>,
            azure_endpoint=<VALUE>,
            azure_ad_token_provider=<VALUE>,
            max_tokens=<VALUE>,
            n=<VALUE>,
            temperature=<VALUE>,
            max_retries=<VALUE>,
            model_kwargs={"top_p": <VALUE>},
        )

And for azure_ad_token_provider, I use Microsoft azure.identity.ClientSecretCredential.get_token. Passing self._get_token to azure_ad_token_provider in AzureChatOpenAI constructor:

from azure.identity import ClientSecretCredential

def __init__(self) -> None:
        self._credential: ClientSecretCredential = ClientSecretCredential(
                tenant_id=<VALUE>,
                client_id=<VALUE>,
                client_secret=<VALUE>,
            )

def _get_token(self) -> str:
        access_token: AccessToken = self._credential.get_token(scope=<VALUE>)
        return access_token.token

[kristapratico]

Thank you for reporting!

How were you initializing the library with langchain? Were you using the Azure version?

Regardless, this does look like a bug to me as well:

openai-python/src/openai/lib/azure.py

Lines 268 to 281 in 50371bf

	def _get_azure_ad_token(self) -> str | None:
	if self._azure_ad_token is not None:
	return self._azure_ad_token
	provider = self._azure_ad_token_provider
	if provider is not None:
	token = provider()
	if not token or not isinstance(token, str): # pyright: ignore[reportUnnecessaryIsInstance]
	raise ValueError(
	f"Expected `azure_ad_token_provider` argument to return a string but it returned {token}",
	)
	return token
	return None

cc @RobertCraigie @kristapratico

@rattrayalex thanks for tagging me! I've opened a PR to fix this in the Azure client: #1531

Also curious, can you expand on what bug you see in the code above?

[RobertCraigie]

this will be fixed in the next release, v1.35.12