The LLM response return should happen after the new Assistant message is stored in the chat memory. #5073
Description
Expected Behavior
When using MessageChatMemoryAdvisor with ChatClient for storing the conversation history the write operation to store the conversation should be completed first and the response generation should happen.
Current Behavior
The write operation of chat conversation history and response generation are running Async and should be synchronous.
How has this issue affected you?
When I amcalling the LLM the response returning doesn't matter to me in my case the response from the LLM will be fetched from the chat memory storage. Once the response is returned and when I try to fetch the message from the chat memory the latest assistant message write operation have not finished and there is a delay of 5-10 seconds in a place where immediate fetching is required.
What are you trying to accomplish?
I am using LLM with workflow and the LLM is guided by the workflow to provide certain message to the user and the workflow guided the LLM as expected and the tried to fetch the assistant message from the chat memory using an API and the latest assistant message is not written into the DB.
What other alternatives have you considered?
Instead of using Advisors I thought of storing the message by calling the add method in the ChatMemory interface but loading the previous conversation history will be affected here.
Are you aware of any workarounds?
In before method of the MessageChatMemoryAdvisor the loading the previous message should happen first and adding the new assisstant message should be done before returning the response or at the time of returning.