Python: Assistant: Callback function on agent.invoke_stream running at the end

[mukeshmrajput]

Problem statement
We have build one ChatCompletionAgent and configure the plugin which actually runs the logic in python. We're invoking agent using the method agent.invoke_stream as we want to streaming out the response, that works fine. Our requirement is to capture the intermediate process in real time when agent is invoking, To achieve this semantic kernel has the concept of call back method to be implemented, we have implemented the same as below

async for response in agent.invoke_stream(messages="user input",
thread=thread,
on_intermediate_message=handle_intermediate_steps):
#Process streamed response(s).
thread = response.thread

Call back function "handle_intermediate_steps" is capturing the intermediate process like FunctionCallContent, FunctionResultContent BUT the main problem here is, it's returning the captured intermediate process at the end of agent.invoke_stream not the real time when agent is invoking.

So wanted to confirm, concept of callback function on agent invoke is designed to run at the end OR real time? If it's support for the real time then it's not working for me even though yield from the semantic kernel function/plugins.

It would be great help if have assistant. I can share the code if needed.

[moonbox3]

@mukeshmrajput please make sure to run 1.30.0. We fixed the bug related to the callback - when using streaming, it will properly handle the interim messages as they’re produced.

[moonbox3]

Closing as of 1.30.0 this callback bug related to the ordering of messages received was fixed.

We have examples in

• code showning this: https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/agents/chat_completion_agent/chat_completion_agent_message_callback_streaming.py
• as well as in our docs: https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/chat-completion-agent?pivots=programming-language-python#handling-intermediate-messages-with-a-chatcompletionagent

[mukeshmrajput]

Thanks @moonbox3. Let me give try with specific version 1.30.0.