-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.Net: Invoking kernel function directly with auto invoke enabled results in multiple nested calls to the function #6281
Comments
In the above code There are a few things that could help here:
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Resources;
namespace GettingStarted;
/// <summary>
/// This example shows how to create a prompt <see cref="KernelFunction"/> from a YAML resource.
/// </summary>
public sealed class Generate_Poem(ITestOutputHelper output) : BaseTest(output)
{
/// <summary>
/// Show how to create a prompt <see cref="KernelFunction"/> from a YAML resource.
/// </summary>
[Fact]
public async Task RunAsync()
{
// Create a kernel with OpenAI chat completion
Kernel kernel = Kernel.CreateBuilder()
.AddOpenAIChatCompletion(
modelId: TestConfiguration.OpenAI.ChatModelId,
apiKey: TestConfiguration.OpenAI.ApiKey)
.Build();
kernel.FunctionInvocationFilters.Add(new FunctionInvocationFilter(Output));
// Load prompt from resource
var generatePoemYaml = EmbeddedResource.Read("GeneratePoem.yaml");
var function = kernel.CreateFunctionFromPromptYaml(generatePoemYaml);
// Add to the Kernel
kernel.Plugins.AddFromFunctions("WriterPlugin", [function]);
// Invoke the prompt function and display the result
OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
// Don't use AutoInvokeKernelFunctions
Console.WriteLine(await kernel.InvokeAsync(function, arguments: new()
{
{ "input", "Write a poem about John Doe" },
}));
Console.WriteLine();
// Call prompt with AutoInvokeKernelFunctions
Console.WriteLine(await kernel.InvokePromptAsync("Write a poem about John Doe", arguments: new(settings)));
}
private sealed class FunctionInvocationFilter(ITestOutputHelper output) : IFunctionInvocationFilter
{
private readonly ITestOutputHelper _output = output;
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
{
this._output.WriteLine($"FunctionInvoking - {context.Function.PluginName}.{context.Function.Name}");
await next(context);
this._output.WriteLine($"FunctionInvoked - {context.Function.PluginName}.{context.Function.Name}");
}
}
} name: GeneratePoem
template: |
Generate a short funny poem or limerick to explain the given event. Be creative and be funny. Let your imagination run wild.
Event:{{$input}}
template_format: semantic-kernel
description: Turn a scenario into a short and entertaining poem.
input_variables:
- name: input
description: The scenario to turn into a poem.
is_required: true
output_variable:
description: The generated poem.
execution_settings:
default:
max_tokens: 60
temperature: 0.5
top_p: 0.0
presence_penalty: 0.0
frequency_penalty: 0.0 |
The sample just shows how to reproduce the issue. In reality, the execution settings with auto invoke enabled could come from somewhere else, such as a service selector. It's fairly easy for someone to have auto invoke enabled by default in their apps and still invoke registered functions using I suggest we have a list of anti-patterns somewhere if we don't suggest users to have auto invoke on and still plan to invoke registered functions directly. |
Say someone has set up a service selector that returns some default settings with auto invoke enabled.
And in their apps, they invoke functions (because nothing suggests they should not). They wouldn't notice anything different but in reality, they could be consuming more tokens they need, and their apps will be slower. |
Agreed. Possibly we shouldn't advertise the current function to the model, I'm not sure if that would ever be useful. We have a task to provide more control over what kernel functions are advertised to the model, I can include this use case as part of that. |
… function (#6544) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Previously inside the telemetry sample app, the code registers a prompt-based plugin to the kernel and then invokes the plugin directly via `InvokeAsync()`. This is actually against the pattern we recommend. The direct result of this is the models with function calling feature could potentially invoke the same function repeatedly. ![image](https://github.com/microsoft/semantic-kernel/assets/12570346/d046e6b7-7220-4af9-aaa0-a198512e2864) This is related to: #6154, #6281 ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Instead of registering the plugin to the kernel for later use, we create the plugin without registering it to the kernel, so that the model doesn't have the same function at its disposal when completing a response. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄
Describe the bug
When a kernel function is invoked directly via
, and with auto invoke enabled, it may result in multiple calls to the function, which means multiple calls to the model, causing unnecessary round trips.
To Reproduce
Steps to reproduce the behavior:
WriterPlugin
to the project root.FunctionInvoking - WriterPlugin.ShortPoem
in the terminal.Expected behavior
A single invocation of the
ShortPoem
function, regardless ifAutoInvokeKernelFunctions
is enabled.With auto invoke disabled, you will see the filter being run only once:
Screenshots
Multiple invocations:
Platform
Additional context
Tested on GPT-4 and GPT-4 turbo
The text was updated successfully, but these errors were encountered: