Azure OpenAI API doesn't work when tools are being used with streamChatCompletions

[jmaczan]

• Package Name: @azure/openai
• Package Version: ^1.0.0-beta.11
• Operating system: Win 10
• nodejs
  • version: v18.12.1
• browser
  • name/version: Chrome 121
• typescript
  • version: 8.19.2
• Is the bug related to documentation in
  • README.md
  • source code documentation
  • SDK API docs on https://docs.microsoft.com

Describe the bug

I try to use tools (a new version of function calling) together with streamChatCompletions() and I get either

{
    requestid: '4b00c2f0-0d49-4faf-9ef0-24bb8c5fd155',
    code: 400,
    message: 'Functions are not supported for this API version or this model version. To learn how to user use function calling with Azure OpenAI Service. Please refer to this wiki  https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/function-calling?tabs=python'
  }

or

  {
    requestid: '5746e80b-3523-44bf-a9db-4e1e7cacc8c9',
    code: 500,
    message: "'dict' object has no attribute 'lower'"
  }

To Reproduce
Steps to reproduce the behavior:

1. Create a GPT-4 deployment in Azure OpenAI Studio with model version 1106-Preview
2. Try to send a request using sdk

import {
    AzureKeyCredential,
    ChatRequestMessage,
    OpenAIClient,
} from "@azure/openai";


try {
    const client = new OpenAIClient(
        process.env.AOAIEndpoint, // fill with your value
        new AzureKeyCredential(process.env.AOAIKey), // fill with your value
        {
            apiVersion: "2023-12-01-preview", // you can try with other values as well
        }
    );

    const deploymentId = process.env.AOAIDeploymentId; // fill with your value

      let result = await client
          .streamChatCompletions(
              deploymentId,
              [{ role: "user", content: "Call my function", }] as ChatRequestMessage[],
              {
                    temperature: 0,
                    topP: 1,
                    maxTokens: 1000,
                    frequencyPenalty: 0,
                    presencePenalty: 0,
                    azureExtensionOptions: {
                        extensions: [
                            {
                                type: "AzureCognitiveSearch",
                                endpoint: process.env.SearchEndpoint,
                                key: process.env.SearchKey,
                                indexName: process.env.SearchIndex,
                                inScope: false,
                                strictness: 2,
                                topNDocuments: 8,
                                semanticConfiguration: "default",
                                queryType: "vectorSemanticHybrid",
                                fieldsMapping: {},
                                roleInformation: roleInformationPrompt,
                                filter: null,
                                embeddingKey: process.env.AOAIKey,
                                embeddingEndpoint: process.env.EmbeddingEndpoint
                                embeddingDependency: {
                                    type: "DeploymentName",
                                    deploymentName: process.env.EmbeddingDeploymentName
                                },
                            },
                        ],
                    },
                  tools: [
                      {
                          function: someFunction,
                          type: "function",
                      },
                  ],
              }
          )

Expected behavior
A clear and concise description of what you expected to happen.

Tool is called when needed

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

[deyaaeldeen]

Thanks for opening this report!

This seems to be a service issue so I'd recommend opening a support ticket in the Azure portal, so our support team can assist you better and resolve it as soon as possible. To open a support ticket, please refer to the guide here.

Also, I wonder, does the same issue manifest when getChatCompletions is being used? Also, does your deployment handle tool calls correctly if azure extensions are not being used?

Thank you for your patience and cooperation. Please let us know if you have any other questions.

[jmaczan]

Hello,

The issue persists for getChatCompletions as well

Removing azureExtensionOptions from API call resolves the issue for both getChatCompletions and streamChatCompletions!

[jmaczan]

Shall I still issue a ticket to Azure? Are you able to confirm that the issue is not on SDK's side?

Much thanks in advance!

[deyaaeldeen]

@jmaczan Yes please, this is definitely a service issue, and @minhanh-phan was able to repro it.

[jmaczan]

Thanks Deyaaeldeen. I have registered an issue in Azure Portal and will let you know about the status when things move on

[jmaczan]

Hey, so after a joint investigation with Izabela from MS, we confirmed that it's an issue with services availability and the (undocumented) fact that some API versions doesn't play well with AI Search. I think we can close the ticket for now, unless you'd like to incorporate some code that aligns API typings with API version, so some parts of API are not available if user uses a particular API version string.

Let me know if I can help further

Thanks and have a great day!

[deyaaeldeen]

Thanks a lot for sharing! In this case, let's close it then.