Integrating Azure OpenAI for Chat in a Docker Setup

[Poltavtcev]

Is there an easy way to integrate Azure OpenAI for chat? I’ve tried several connection options, but haven’t been able to get it working. I’m using a Docker setup and don’t have much flexibility to make significant modifications to the product itself. It would be great to have the option to integrate through a configuration file and environment variables. Has anyone managed to get Azure OpenAI working?

[DavidMStraub]

Does Azure OpenAI work with the openai Python library?

[Poltavtcev]

Yes, Azure OpenAI works with the OpenAI Python library. Here is the documentation on how to set it up:

Microsoft Azure OpenAI

To use this library with Azure OpenAI, use the AzureOpenAI
class instead of the OpenAI class.

Important

The Azure API shape differs from the core API shape which means that the static types for responses / params
won't always be correct.

from openai import AzureOpenAI

# gets the API Key from environment variable AZURE_OPENAI_API_KEY
client = AzureOpenAI(
    # https://learn.microsoft.com/azure/ai-services/openai/reference#rest-api-versioning
    api_version="2023-07-01-preview",
    # https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource
    azure_endpoint="https://example-endpoint.openai.azure.com",
)

completion = client.chat.completions.create(
    model="deployment-name",  # e.g. gpt-35-instant
    messages=[
        {
            "role": "user",
            "content": "How do I output all files in a directory using Python?",
        },
    ],
)
print(completion.to_json())

In addition to the options provided in the base OpenAI client, the following options are provided:

• azure_endpoint (or the AZURE_OPENAI_ENDPOINT environment variable)
• azure_deployment
• api_version (or the OPENAI_API_VERSION environment variable)
• azure_ad_token (or the AZURE_OPENAI_AD_TOKEN environment variable)
• azure_ad_token_provider

An example of using the client with Microsoft Entra ID (formerly known as Azure Active Directory) can be found here.

[DavidMStraub]

Ok, so that would take quite some manual adaption. If somebody is willing to contribute it, we can add it.

[Poltavtcev]

Understood. No, I don’t think this feature is necessary enough to spend time on. I thought it might already work out of the box using the library’s capabilities. However, at the moment, the AI chat itself is quite limited in functionality. Yes, it’s an interesting and fun feature. But how to make the most of it—I can’t even imagine for now…