Description
Describe the bug
When making a request to an agent via the AI Agent Service that is grounded with Bing, InvokeStreamingAsync() throws an exception after the last message is rendered. This appears to be related to the citation.
To Reproduce
Test Code
[Fact]
[Experimental("OPENAI001")]
public async Task InternetSearchFlattened()
{
var foundryConfig = GetService<AiFoundryConfig>();
var client = AzureAIAgent.CreateAzureAIClient(foundryConfig.ConnectionString, new DefaultAzureCredential());
var agentsClient = client.GetAgentsClient();
var getAgentResponse = await agentsClient.GetAgentAsync(AgentId);
AzureAIAgent agent = new(getAgentResponse.Value, agentsClient);
AgentThread thread = new AzureAIAgentThread(agentsClient);
await foreach (var responseUpdate in agent.InvokeStreamingAsync("What are today's news headlines", thread))
{
Logger.Debug("Streaming chunk: {Chunk}", responseUpdate.Message.Content);
}
}
Exception
System.NullReferenceException
Object reference not set to an instance of an object.
at Azure.AI.Projects.TextAnnotationUpdate.get_TextToReplace()
at Microsoft.SemanticKernel.Agents.AzureAI.Internal.AgentThreadActions.GenerateStreamingAnnotationContent(TextAnnotationUpdate annotation)
at Microsoft.SemanticKernel.Agents.AzureAI.Internal.AgentThreadActions.GenerateStreamingMessageContent(String assistantName, MessageContentUpdate update)
at Microsoft.SemanticKernel.Agents.AzureAI.Internal.AgentThreadActions.InvokeStreamingAsync(AzureAIAgent agent, AgentsClient client, String threadId, IList`1 messages, AzureAIInvocationOptions invocationOptions, ILogger logger, Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Agents.AzureAI.Internal.AgentThreadActions.InvokeStreamingAsync(AzureAIAgent agent, AgentsClient client, String threadId, IList`1 messages, AzureAIInvocationOptions invocationOptions, ILogger logger, Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Agents.AzureAI.Internal.AgentThreadActions.InvokeStreamingAsync(AzureAIAgent agent, AgentsClient client, String threadId, IList`1 messages, AzureAIInvocationOptions invocationOptions, ILogger logger, Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at Microsoft.SemanticKernel.Diagnostics.ActivityExtensions.RunWithActivityAsync[TResult](Func`1 getActivity, Func`1 operation, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Diagnostics.ActivityExtensions.RunWithActivityAsync[TResult](Func`1 getActivity, Func`1 operation, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Diagnostics.ActivityExtensions.RunWithActivityAsync[TResult](Func`1 getActivity, Func`1 operation, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at Microsoft.SemanticKernel.Agents.AzureAI.AzureAIAgent.InvokeStreamingAsync(ICollection`1 messages, AgentThread thread, AzureAIAgentInvokeOptions options, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Agents.AzureAI.AzureAIAgent.InvokeStreamingAsync(ICollection`1 messages, AgentThread thread, AzureAIAgentInvokeOptions options, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Agents.AzureAI.AzureAIAgent.InvokeStreamingAsync(ICollection`1 messages, AgentThread thread, AzureAIAgentInvokeOptions options, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at SKSpikes.Experiments.AgentChat.Agent004AzureAiServicesAgent.InternetSearchFlattened() in C:\_cfdev\semantic-kernel-spikes\Experiments\AgentChat\Agent004AzureAiServicesAgent.cs:line 208
at SKSpikes.Experiments.AgentChat.Agent004AzureAiServicesAgent.InternetSearchFlattened() in C:\_cfdev\semantic-kernel-spikes\Experiments\AgentChat\Agent004AzureAiServicesAgent.cs:line 208
at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass47_0.<<InvokeTestMethodAsync>b__1>d.MoveNext() in /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs:line 259
--- End of stack trace from previous location ---
at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs:line 48
at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in /_/src/xunit.core/Sdk/ExceptionAggregator.cs:line 90
Debug Output:
ℹ️ 08:04:50.230 Test "Agent004AzureAiServicesAgent" initialized
🐞 08:04:59.948 Streaming chunk: "Today's"
🐞 08:04:59.949 Streaming chunk: " news"
🐞 08:04:59.949 Streaming chunk: " headlines"
🐞 08:04:59.950 Streaming chunk: " include"
🐞 08:05:00.000 Streaming chunk: ":
"
🐞 08:05:00.126 Streaming chunk: "1"
🐞 08:05:00.126 Streaming chunk: "."
🐞 08:05:00.127 Streaming chunk: " Hamas"
🐞 08:05:00.127 Streaming chunk: " negoti"
🐞 08:05:00.127 Streaming chunk: "ators"
🐞 08:05:00.128 Streaming chunk: " are"
🐞 08:05:00.128 Streaming chunk: " considering"
🐞 08:05:00.128 Streaming chunk: " a"
🐞 08:05:00.128 Streaming chunk: " cease"
🐞 08:05:00.128 Streaming chunk: "fire"
🐞 08:05:00.128 Streaming chunk: " proposal"
🐞 08:05:00.128 Streaming chunk: " from"
🐞 08:05:00.129 Streaming chunk: " Israel"
🐞 08:05:00.150 Streaming chunk: ","
🐞 08:05:00.192 Streaming chunk: " mediated"
🐞 08:05:00.192 Streaming chunk: " by"
🐞 08:05:00.192 Streaming chunk: " Egyptian"
🐞 08:05:00.192 Streaming chunk: " and"
🐞 08:05:00.193 Streaming chunk: " Q"
🐞 08:05:00.193 Streaming chunk: "at"
🐞 08:05:00.253 Streaming chunk: "ari"
🐞 08:05:00.316 Streaming chunk: " representatives"
🐞 08:05:00.316 Streaming chunk: ","
🐞 08:05:00.316 Streaming chunk: " which"
🐞 08:05:00.316 Streaming chunk: " includes"
🐞 08:05:00.316 Streaming chunk: " demands"
🐞 08:05:00.316 Streaming chunk: " for"
🐞 08:05:00.316 Streaming chunk: " Hamas"
🐞 08:05:00.316 Streaming chunk: " to"
🐞 08:05:00.366 Streaming chunk: " dis"
🐞 08:05:00.366 Streaming chunk: "arm"
🐞 08:05:00.459 Streaming chunk: "【3:0†source】"
Expected behavior
No exception thrown
Other Information
This only occurs with when the Bing search tool is invoked - ie, a request to "Why is the sky blue?" will not throw an exception.
Platform
- Language: C#
- Source: MS.SK.Agents.AzureAI .46.0-preview . & MS.SK 1.40
- AI model: gpt4o (2024-08-06)
- IDE: Rider
- OS: Windows
Metadata
Metadata
Assignees
Type
Projects
Status