Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net Issue Fix - Add FunctionResponse to ChatStreaming interfaces #3246

Merged
merged 11 commits into from
Nov 1, 2023

Conversation

RogerBarreto
Copy link
Member

Motivation and Context

Resolves #3198

Today we can't call functions using streaming, this change allows it to be used.

The current approach will require the stream to be buffered while listening to a potential function call.

The example shows how that will be achieved.

StringBuilder chatContent = new();

// Non function result streaming will happen here.
await foreach (var message in chatResult.GetStreamingChatMessageAsync())
{
    if (message.Content is not null)
    {
        Console.Write(message.Content);
        chatContent.Append(message.Content);
    }
}
chatHistory.AddAssistantMessage(chatContent.ToString());

// After all stream was loaded check if a `FunctionResponse` was present.
var functionResponse = await chatResult.GetStreamingFunctionResponseAsync();

Remark: Calling GetStreamingFunctionResponseAsync() will buffer the stream to capture the full function call definition, use it after all the streaming messages were flushed out.

Description

Allows streaming interfaces to get the function result.

This is a simplified approach (Buffer the streaming of a function call request until it's complete). Using the FunctionCall

Contribution Checklist

@RogerBarreto RogerBarreto added enhancement PR: feedback to address Waiting for PR owner to address comments/questions PR: ready for review All feedback addressed, ready for reviews .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel labels Oct 20, 2023
@RogerBarreto RogerBarreto self-assigned this Oct 20, 2023
@RogerBarreto RogerBarreto requested a review from a team as a code owner October 20, 2023 13:21
@RogerBarreto RogerBarreto removed the PR: feedback to address Waiting for PR owner to address comments/questions label Oct 20, 2023
@RogerBarreto RogerBarreto added the PR: ready to merge PR has been approved by all reviewers, and is ready to merge. label Nov 1, 2023
@RogerBarreto RogerBarreto added this pull request to the merge queue Nov 1, 2023
Merged via the queue into microsoft:main with commit 831ff8e Nov 1, 2023
19 checks passed
@RogerBarreto RogerBarreto deleted the issues/fix-3198-2730-2 branch November 1, 2023 19:07
@alexminza
Copy link

@RogerBarreto it seems that when the chat completion call returns just text and no function call to be invoked - the current code will just throw a NotSupportedException when checking GetOpenAIStreamingFunctionResponseAsync because of https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/Connectors/Connectors.AI.OpenAI/AzureSdk/SKChatMessage.cs#L40

public FunctionCall FunctionCall
        => this._message?.FunctionCall ?? throw new NotSupportedException("Function call is not supported");

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code PR: ready for review All feedback addressed, ready for reviews PR: ready to merge PR has been approved by all reviewers, and is ready to merge.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

.Net: GetFunctionResponse doesn't work with IChatStreamingResult (and can't)
5 participants