Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net: Added support for multiple chat and text results from Kernel #6704

Merged
merged 14 commits into from
Jun 18, 2024

Conversation

dmytrostruk
Copy link
Member

@dmytrostruk dmytrostruk commented Jun 13, 2024

Motivation and Context

Note: This PR changes the behavior when AI connector returns multiple results, when using kernel.InvokeAsync. It won't throw an exception anymore, but instead it will return multiple results. The behavior for single result is not changed.

Fixes: #6434

When executing prompt function using kernel and asking for multiple results per prompt, we will get an error:

var arguments = new KernelArguments(new OpenAIPromptExecutionSettings
{
    ResultsPerPrompt = 3
});

var result = await this._kernel.InvokePromptAsync("Hi, can you help me today?", arguments); // this will throw an exception

Current KernelFunctionFromPrompt implementation expects only single result from AI connector, while its API can return multiple results per prompt/request.

This PR updates KernelFunctionFromPrompt in a following way:

  1. If AI connector returns single item - the behavior will be the same as it is today, FunctionResult will contain instance of that item, so it's possible to get its properties, use ToString() etc.
  2. If AI connector returns multiple items - all items will be returned in collection to the caller, and this collection needs to be handled appropriately (by using loop or accessing specific item by index).

One of the examples shows how to select one result, in case if we invoke prompt function inside another prompt function using prompt template engine. In this case, filter can be registered, which will get multiple results produced by function, select one of them and return it back to the prompt rendering operation.

Contribution Checklist

@dmytrostruk dmytrostruk self-assigned this Jun 13, 2024
@dmytrostruk dmytrostruk requested a review from a team as a code owner June 13, 2024 22:42
@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel kernel.core labels Jun 13, 2024
@RogerBarreto RogerBarreto added the PR: breaking change Pull requests that introduce breaking changes label Jun 14, 2024
@dmytrostruk dmytrostruk removed the PR: breaking change Pull requests that introduce breaking changes label Jun 14, 2024
@dmytrostruk dmytrostruk added the PR: breaking change Pull requests that introduce breaking changes label Jun 17, 2024
@dmytrostruk dmytrostruk added this pull request to the merge queue Jun 18, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 18, 2024
@dmytrostruk dmytrostruk added this pull request to the merge queue Jun 18, 2024
Merged via the queue into microsoft:main with commit 4b8a526 Jun 18, 2024
15 checks passed
@dmytrostruk dmytrostruk deleted the multiple-text-chat-results branch June 18, 2024 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel.core kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code PR: breaking change Pull requests that introduce breaking changes
Projects
None yet
4 participants