HandlebarsPlan to use custom httpclient and httpclient handler when invoke ( handlerbarsplan.invokeAsync(kernel, initialContext)

[DhirendraPatil]

Hi,

I am trying to the Example65 of coursera courses. I was able to intercept add custom httphandler when the plan is created however when the actual plan was invoked which calls coursera api, the custom httphandler which I setup for kernel does not get invoke. looks like another httpclient and httphandler gets used?

can you clarify?

[sophialagerkranspandey]

Could you please share the code you're using?

[DhirendraPatil]

dotnet/samples/KernelSyntaxExamples/Example65_HandlebarsPlanner.cs

Above sample was used.
Instead of azureOpenAIChatCompletion,

OpenAIChatCompletion is used and passed httpclient which uses custom httphandler.

RunCourseraSampleAsync used for this test.
The observation is that custom httphandler injected in Kernal via builder, used only for actual openAi call however when handlerbarplan actually invokes, the custom httphandler does not get invoked.

[DhirendraPatil]

@sophialagerkranspandey see my response below with sample. Let me know if u need anything.

When the handlebar makes actual call to coursera, the http call does not uses the customhttphandler in this case

[DhirendraPatil]

Example65_HandlebarsPlanner handlebar = new Example65_HandlebarsPlanner();
await handlebar.RunCourseraSampleAsync(true);
//=========
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.Planning;
using Microsoft.SemanticKernel.Planning.Handlebars;
using Microsoft.SemanticKernel.Plugins.OpenApi;
using Plugins.DictionaryPlugin;
using RepoUtils;
using xRetry;

using static System.Console;
namespace Examples;

public class CustomHttpHanadler : HttpClientHandler
{
protected override HttpResponseMessage Send(HttpRequestMessage request, CancellationToken cancellationToken)
{
return base.Send(request, cancellationToken);
}

protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
    Console.WriteLine("Request \n" + request.RequestUri + " \n" + request.Content.ReadAsStringAsync().Result + "\n");
    return base.SendAsync(request, cancellationToken);

}

}

public class Example65_HandlebarsPlanner
{
private static int s_sampleIndex;

private const string CourseraPluginName = "CourseraPlugin";
private static String modelId = "gpt-4";
private static string apiKey = "apikey";
private void WriteSampleHeading(string name)
{
    WriteLine($"======== [Handlebars Planner] Sample {s_sampleIndex++} - Create and Execute Plan with: {name} ========");
}

private async Task<Kernel?> SetupKernelAsync(params string[] pluginDirectoryNames)
{

    var handler = new CustomHttpHanadler();
    HttpClient client = new HttpClient(handler);
    var kernel = Kernel.CreateBuilder()
        .AddOpenAIChatCompletion(modelId, apiKey, httpClient: client)
        .Build();


    if (pluginDirectoryNames.Length > 0)
    {
        if (pluginDirectoryNames[0] == StringParamsDictionaryPlugin.PluginName)
        {
            kernel.ImportPluginFromType<StringParamsDictionaryPlugin>(StringParamsDictionaryPlugin.PluginName);
        }
        else if (pluginDirectoryNames[0] == ComplexParamsDictionaryPlugin.PluginName)
        {
            kernel.ImportPluginFromType<ComplexParamsDictionaryPlugin>(ComplexParamsDictionaryPlugin.PluginName);
        }
        else if (pluginDirectoryNames[0] == CourseraPluginName)
        {
            await kernel.ImportPluginFromOpenApiAsync(
                CourseraPluginName,
                new Uri("https://www.coursera.org/api/rest/v1/search/openapi.yaml")
            );
        }
        else
        {
            string folder = RepoFiles.SamplePluginsPath();

            foreach (var pluginDirectoryName in pluginDirectoryNames)
            {
                kernel.ImportPluginFromPromptDirectory(Path.Combine(folder, pluginDirectoryName));
            }
        }
    }

    return kernel;
}

private void PrintPlannerDetails(string goal, HandlebarsPlan plan, string result, bool shouldPrintPrompt)
{
    WriteLine($"Goal: {goal}");
    WriteLine($"\nOriginal plan:\n{plan}");
    WriteLine($"\nResult:\n{result}\n");

    // Print the prompt template
    if (shouldPrintPrompt && plan.Prompt is not null)
    {
        WriteLine("\n======== CreatePlan Prompt ========");
        WriteLine(plan.Prompt);
    }
}

private async Task RunSampleAsync(
    string goal,
    HandlebarsPlannerOptions? plannerOptions = null,
    KernelArguments? initialContext = null,
    bool shouldPrintPrompt = false,
    bool shouldInvokePlan = true,
    params string[] pluginDirectoryNames)
{
    var kernel = await SetupKernelAsync(pluginDirectoryNames);
    if (kernel is null)
    {
        return;
    }

    // Set the planner options
    plannerOptions ??= new HandlebarsPlannerOptions()
    {
        // When using OpenAI models, we recommend using low values for temperature and top_p to minimize planner hallucinations.
        ExecutionSettings = new OpenAIPromptExecutionSettings()
        {
            Temperature = 0.0,
            TopP = 0.1,
        },
    };
    FunctionCallingStepwisePlanner functionCallingStep = new FunctionCallingStepwisePlanner();

    // Use gpt-4 or newer models if you want to test with loops.
    // Older models like gpt-35-turbo are less recommended. They do handle loops but are more prone to syntax errors.
    plannerOptions.AllowLoops = true;

    // Instantiate the planner and create the plan
    var planner = new HandlebarsPlanner(plannerOptions);
    var plan = await planner.CreatePlanAsync(kernel, goal, initialContext);

    // Execute the plan
    var result = shouldInvokePlan ? await plan.InvokeAsync(kernel, initialContext) : string.Empty;

    PrintPlannerDetails(goal, plan, result, shouldPrintPrompt);
}
[RetryTheory(typeof(HttpOperationException))]
[InlineData(true)]
public Task RunCourseraSampleAsync(bool shouldPrintPrompt)
{
    WriteSampleHeading("Coursera OpenAPI Plugin");
    return RunSampleAsync("Show me courses about Artificial Intelligence.", null, null, shouldPrintPrompt, true, CourseraPluginName);
    /*
        Original plan:
        {{!-- Step 0: Extract key values --}}
        {{set "query" "Artificial Intelligence"}}

        {{!-- Step 1: Call CourseraPlugin-search with the query --}}
        {{set "searchResults" (CourseraPlugin-search query=query)}}

        {{!-- Step 2: Loop through the search results and display course information --}}
        {{#each searchResults.hits}}
            {{json (concat "Course Name: " this.name ", URL: " this.objectUrl)}}
        {{/each}}

        Result:
        Course Name: Introduction to Artificial Intelligence (AI), URL: https://www.coursera.org/learn/introduction-to-ai?utm_source=rest_api
        Course Name: IBM Applied AI, URL: https://www.coursera.org/professional-certificates/applied-artifical-intelligence-ibm-watson-ai?utm_source=rest_api
        Course Name: AI For Everyone, URL: https://www.coursera.org/learn/ai-for-everyone?utm_source=rest_api
        Course Name: Python for Data Science, AI & Development, URL: https://www.coursera.org/learn/python-for-applied-data-science-ai?utm_source=rest_api
        Course Name: Introduction to Generative AI, URL: https://www.coursera.org/learn/introduction-to-generative-ai?utm_source=rest_api
        Course Name: Deep Learning, URL: https://www.coursera.org/specializations/deep-learning?utm_source=rest_api
        Course Name: Machine Learning, URL: https://www.coursera.org/specializations/machine-learning-introduction?utm_source=rest_api
        Course Name: IBM AI Engineering, URL: https://www.coursera.org/professional-certificates/ai-engineer?utm_source=rest_api

    */
}

}

[matthewbolanos]

The reason the HTTP client is not being used for Coursera is because it's only being attached to the Chat Completion service. If you would like to use the same HTTP client for Coursera, then you would need to also pass the HTTP client to the Coursera plugin using the optional http client input parameter.

The reason why these are separate is because you often want different HTTP clients for handling different scenarios like auth.