Specify OpenAI Execution settings per Invoke

[LefanTan]

Hi all!

Currently, we have an internal tool that needs a way to allow users to specify their preferred GPT model (as well as other configs such as Temperature).

It seems like in SK, the Models are pre-defined in builder.AddOpenAIChatCompletion(_appSettings.App.SemanticKernel.TextModel, chatCompletionClient); as well as config.json. I'm currently using kernel.InvokeAsync but may be using a planner soon too. How can I approach this?

[dmytrostruk]

Hi @LefanTan , it's possible to set OpenAI execution settings per Invoke by passing settings model within KernelArguments. Here is the syntax:

semantic-kernel/dotnet/samples/KernelSyntaxExamples/Example58_ConfigureExecutionSettings.cs

Lines 47 to 54 in 9b8a218

	var result = await kernel.InvokePromptAsync(
	prompt,
	new(new OpenAIPromptExecutionSettings()
	{
	MaxTokens = 60,
	Temperature = 0.7
	}));
	this.WriteLine(result.GetValue<string>());

Please let me know if that resolves your scenario. Thank you!

[LefanTan]

This is perfect, thank you!

[LefanTan]

@dmytrostruk Upon further inspection, it seems like the ModelId defined during builder.AddOpenAIChatCompletion(modelId)
will override whatever specified in KernelArguments

Example:

[dmytrostruk]

@LefanTan That's because you need to register multiple instances of OpenAI connector per model, like in this example:

// Build a kernel with multiple chat completion services
var builder = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion(
        modelId: "gpt-4-0125-preview",
        apiKey: TestConfiguration.OpenAI.ApiKey)
    .AddOpenAIChatCompletion(
        modelId: "gpt-3.5-turbo-0125",
        apiKey: TestConfiguration.OpenAI.ApiKey);

Kernel kernel = builder.Build();

var prompt = "Hello AI, what can you do for me?";

var result = await kernel.InvokePromptAsync(prompt, new(new OpenAIPromptExecutionSettings { ModelId = "gpt-3.5-turbo-0125" }));

In this case, it will use the model specified in OpenAIPromptExecutionSettings.ModelId.

It is also possible to inject your own IAIServiceSelector implementation, which will select AI service based on your conditions. Here is an example with custom AI service selector:
https://github.com/microsoft/semantic-kernel/blob/57b55af5786ed44630896b5ad4916636c3deba8e/dotnet/samples/KernelSyntaxExamples/Example62_CustomAIServiceSelector.cs