forked from microsoft/semantic-kernel
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
.Net: New result types - FunctionResult and KernelResult (microsoft#2864
) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR contains changes to replace `SKContext` as result type with new types `FunctionResult` and `KernelResult`. 1. `FunctionResult` - new return type from single function invocation. It contains `object? Value` which can be any primitive, complex type or `IAsyncEnumerable<T>`. It also contains `SKContext`, but internally. This is required to pass `SKContext` to the next function in pipeline and keep the implementation simple at first iteration. Context could be removed from `FunctionResult` if needed in the future. It's not publicly available as part of `FunctionResult`. 2. `KernelResult` - new return type in `Kernel.RunAsync` method. It doesn't contain `SKContext`, just `object? Value` to get execution result. `FunctionResult` also contains `Metadata` - property, that contains additional data, including AI model response (e.g. token usage). `KernelResult` contains collection of `FunctionResult` - all function results from pipeline, so it's possible to check result of each function in pipeline, observe result value, AI-related information, like amount of tokens etc. Syntax in examples and tests were changed from `result.Result` to `result.GetValue<string>`, but now it should be possible to get any type as a result and not only string. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> 1. Updated `ISKFunction` interface to return `FunctionResult` instead of `SKContext`. 2. Updated `IKernel` interface to return `KernelResult` instead of `SKContext`. 3. Updated `NativeFunction` to support `IAsyncEnumerable<T>` return type. 4. Added tests to `SKFunctionTests2` to verify the behavior for different result types. 5. Removed test `KernelTests.ItProvidesAccessToFunctionsViaSKContextAsync`. Reason: Invalid scenario. Kernel should not provide public access to functions via `SKContext`. 6. Updated tests and examples across the codebase to align with new signature. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 - BREAKING CHANGE: This PR will modify syntax of getting result using `ISKFunction.InvokeAsync` and `IKernel.RunAsync` methods. Syntax changed from `SKContext.Result` to `FunctionResult/KernelResult.GetValue<T>()`.
- Loading branch information
1 parent
bf9c1fc
commit 544b6c1
Showing
58 changed files
with
967 additions
and
419 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
--- | ||
# These are optional elements. Feel free to remove any of them. | ||
status: accepted | ||
date: 2023-09-21 | ||
deciders: shawncal, dmytrostruk | ||
consulted: | ||
informed: | ||
--- | ||
# Replace SKContext as Function/Kernel result type with FunctionResult and KernelResult models | ||
|
||
## Context and Problem Statement | ||
|
||
Methods `function.InvokeAsync` and `kernel.RunAsync` return `SKContext` as result type. This has several problems: | ||
|
||
1. `SKContext` contains property `Result`, which is `string`. Based on that, it's not possible to return complex type or implement streaming capability in Kernel. | ||
2. `SKContext` contains property `ModelResults`, which is coupled to LLM-specific logic, so it's only applicable to semantic functions in specific cases. | ||
3. `SKContext` as a mechanism of passing information between functions in pipeline should be internal implementation. Caller of Kernel should provide input/request and receive some result, but not `SKContext`. | ||
4. `SKContext` contains information related to the last executed function without a way to access information about specific function in pipeline. | ||
|
||
## Decision Drivers | ||
|
||
1. Kernel should be able to return complex type as well as support streaming capability. | ||
2. Kernel should be able to return data related to function execution (e.g. amount of tokens used) in a way, when it's not coupled to AI logic. | ||
3. `SKContext` should work as internal mechanism of passing information between functions. | ||
4. There should be a way how to differentiate function result from kernel result, since these entities are different by nature and may contain different set of properties in the future. | ||
5. The possibility to access specific function result in the middle of pipeline will provide more insights to the users how their functions performed. | ||
|
||
## Considered Options | ||
|
||
1. Use `dynamic` as return type - this option provides some flexibility, but on the other hand removes strong typing, which is preferred option in .NET world. Also, there will be no way how to differentiate function result from Kernel result. | ||
2. Define new types - `FunctionResult` and `KernelResult` - chosen approach. | ||
|
||
## Decision Outcome | ||
|
||
New `FunctionResult` and `KernelResult` return types should cover scenarios like returning complex types from functions, supporting streaming and possibility to access result of each function separately. | ||
|
||
### Complex Types and Streaming | ||
|
||
For complex types and streaming, property `object Value` will be defined in `FunctionResult` to store single function result, and in `KernelResult` to store result from last function in execution pipeline. For better usability, generic method `GetValue<T>` will allow to cast `object Value` to specific type. | ||
|
||
Examples: | ||
|
||
```csharp | ||
// string | ||
var text = (await kernel.RunAsync(function)).GetValue<string>(); | ||
|
||
// complex type | ||
var myComplexType = (await kernel.RunAsync(function)).GetValue<MyComplexType>(); | ||
|
||
// streaming | ||
var results = (await kernel.RunAsync(function)).GetValue<IAsyncEnumerable<int>>(); | ||
|
||
await foreach (var result in results) | ||
{ | ||
Console.WriteLine(result); | ||
} | ||
``` | ||
|
||
When `FunctionResult`/`KernelResult` will store `TypeA` and caller will try to cast it to `TypeB` - in this case `InvalidCastException` will be thrown with details about types. This will provide some information to the caller which type should be used for casting. | ||
|
||
### Metadata | ||
|
||
To return additional information related to function execution - property `Dictionary<string, object> Metadata` will be added to `FunctionResult`. This will allow to pass any kind of information to the caller, which should provide some insights how function performed (e.g. amount of tokens used, AI model response etc.) | ||
|
||
Examples: | ||
|
||
```csharp | ||
var functionResult = await function.InvokeAsync(context); | ||
Console.WriteLine(functionResult.Metadata["MyInfo"]); | ||
``` | ||
|
||
### Multiple function results | ||
|
||
`KernelResult` will contain collection of function results - `IReadOnlyCollection<FunctionResult> FunctionResults`. This will allow to get specific function result from `KernelResult`. Properties `FunctionName` and `PluginName` in `FunctionResult` will help to get specific function from collection. | ||
|
||
Example: | ||
|
||
```csharp | ||
var kernelResult = await kernel.RunAsync(function1, function2, function3); | ||
|
||
var functionResult2 = kernelResult.FunctionResults.First(l => l.FunctionName == "Function2" && l.PluginName == "MyPlugin"); | ||
|
||
Assert.Equal("Result2", functionResult2.GetValue<string>()); | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.