.Net: Bug: OnIntermediateMessage callback in InvokeStreamingAsync is triggered only after streaming completes when using plugins

[Macro21]

Describe the bug
When using InvokeStreamingAsync with an agent that calls plugins (functions), the OnIntermediateMessage callback is not triggered during the intermediate tool execution phase. Instead, it is only invoked after the entire streaming response is completed. This behavior differs from InvokeAsync, where OnIntermediateMessage is triggered as soon as the tool response is available.

This makes it impossible to stream tool/function results to the client before the final response is generated, which is critical for real-time feedback in chat applications.

To Reproduce
Steps to reproduce the behavior:

Create an agent that invokes one or more plugins (functions).
Use InvokeStreamingAsync with an AgentInvokeOptions that includes an OnIntermediateMessage callback.
Observe that the callback is not triggered when the tool is executed, but only after the full response is streamed.

ChatMessageContent input = new(AuthorRole.User, userPrompt);

var corporateAgent = agentFactory.Get(AgentName.CorporateAgent);

AgentThread agentThread = new ChatHistoryAgentThread(chatHistory);

var agentInvokeOptions = new AgentInvokeOptions()
{
    OnIntermediateMessage = async (message) =>
    {
        var partialResponse = new StreamingResponse
        {
            Id = systemMessageEntity.Id,
            Content = contentBuilder.ToString()
        };

        if (!string.IsNullOrEmpty(message.Content) && message.Role == AuthorRole.Tool)
        {
            var citations = JsonConvert.DeserializeObject<List<ChatCitation>>(message.Content) ?? [ ];
            partialResponse.Citations = citations;

            var settings = new JsonSerializerSettings
            {
                ContractResolver = new CamelCasePropertyNamesContractResolver()
            };

            var data = JsonConvert.SerializeObject(partialResponse, settings);

            await response.WriteAsync($"data: {data}\n\n", cancellationToken);
            await response.Body.FlushAsync(cancellationToken);
        }
    }
};

 var agentStreamingResult = corporateAgent.InvokeStreamingAsync(input, agentThread, agentInvokeOptions, cancellationToken);

 await foreach (var streamingUpdate in agentStreamingResult)
 {
      contentBuilder.Append(streamingUpdate.Message.Content);

      var settings = new JsonSerializerSettings
      {
           ContractResolver = new CamelCasePropertyNamesContractResolver()
      };

      var partialResponse = new StreamingResponse
      {
           Id = systemMessageEntity.Id,
           Content = contentBuilder.ToString(),
      };

      var data = JsonConvert.SerializeObject(partialResponse, settings);

      await response.WriteAsync($"data: {data}\n\n", cancellationToken);
      await response.Body.FlushAsync(cancellationToken);
      await Task.Delay(50, cancellationToken);
 }

 var result = contentBuilder.ToString();

Expected behavior
The OnIntermediateMessage callback should be triggered immediately when a plugin/tool response is available, before the final streaming response is generated. This would allow partial tool results to be sent to the client in real time.

Platform
Language: C#
Source: NuGet package version 1.57.0
AI model: Azure OpenAI:GPT-4o-mini(2024-07-18)
IDE: Visual Studio 2022
OS: Windows 11

Additional context
This issue affects scenarios where multiple tools are invoked before the final response is generated. The inability to stream intermediate tool results limits the responsiveness of the application and breaks consistency with InvokeAsync behavior.