[BUG] Azure.AI.OpenAI beta.12: AzureExtensionsContext.Messages doesn't return citations since Jan. 12th, 2024

[freistli]

Library name and version

Azure.AI.OpenAI beta.12

Describe the bug

With below code:

 Response<ChatCompletions> response = await client.GetChatCompletionsAsync(chatCompletionsOptions);

 ChatResponseMessage responseMessage = response.Value.Choices[0].Message;

responseMessage.Content includes correct responses from GPT-4 turbo, but responseMessage.AzureExtensionsContext.Messages doesn't have any citation now.

The problem starts happening since Jan, 12th, 2024, without code changing.

Before this problem happens, AOAI caused APIKey error from Jan. 11st 2024, but after around 48 hours it was recovered by itself. The bug was reported in #41138

This bug seems still related to AOAI changes (unknown) because code and library are the same.

Expected behavior

AzureExtensionsContext.Messages contains citations info as before

Actual behavior

The count in AzureExtensionsContext.Messages.Citations is 0 even the response is meaningful and have reference numbers [docN]

Reproduction Steps

 Response<ChatCompletions> response = await client.GetChatCompletionsAsync(chatCompletionsOptions);

 ChatResponseMessage responseMessage = response.Value.Choices[0].Message;

if (responseMessage.AzureExtensionsContext.Messages?.Count > 0)
 {
    ...
}
else
{
Console.WriteLine("No citations or other information.");
}

Environment

AOAI service,
GPT-4 turbo,
West US location,
Azure.AI.OpenAI beta.12 nuget.

[LucGosso]

Reproduction Steps, Complete example with azure search endpoints

` AzureCognitiveSearchChatExtensionConfiguration extensionConfig = new()
{
SearchEndpoint = new Uri(_configuration["AzureSearchServiceEndpoint"] + ""),
Authentication = new OnYourDataApiKeyAuthenticationOptions(_configuration["AzureSearchServiceApiKey"]),
IndexName = _configuration["AzureSearchIndex"],
DocumentCount = 10,
//Key = _configuration["AzureSearchServiceApiKey"],
/FieldMappingOptions = new AzureCognitiveSearchIndexFieldMappingOptions()
{
ContentFieldNames = { "content" },
TitleFieldName = "metadata_storage_name",
UrlFieldName = "url",
FilepathFieldName = "metadata_storage_name"
}/
};
var chatCompletionsOptions = new ChatCompletionsOptions()
{
DeploymentName = _configuration["AzureOpenAiChatGptDeployment"],
Messages =
{
//new ChatRequestSystemMessage(promptTemplate),
new ChatRequestUserMessage(question),
},
// The addition of AzureChatExtensionsOptions enables the use of Azure OpenAI capabilities that add to
// the behavior of Chat Completions, here the "using your own data" feature to supplement the context
// with information from an Azure AI Search resource with documents that have been indexed.
AzureExtensionsOptions = new AzureChatExtensionsOptions()
{
Extensions = { extensionConfig }
}
};

Response response = await client.GetChatCompletionsAsync(chatCompletionsOptions);

ChatResponseMessage responseMessage = response.Value.Choices[0].Message;

if (responseMessage.AzureExtensionsContext.Messages?.Count > 0)
{
...
}
else
{
Console.WriteLine("No citations or other information.");
}
`

[thesnake666]

Yes, confirm, the result is ok for content but the citations field is null

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.

[andrej-melicher]

same issue here, as seen on debug screenshot

when using REST API in Postman against same resource, the response is fine, including citations

[freistli]

The issue was auto recovered just now. I can get citations again with the same .Net nuget.

Even though, the .Net version still needs to fix this as REST API or Python client were not impacted by the same issue at the same time even AOAI experienced some unknown behavior changes in past 24 hours.

We don't know when AOAI runs into the same situation and impacts .Net nuget again.

The same suggestion to #41138

Thanks.