How to improve the performance while using Native Functions Plugins with AutoInvokeKernelFunctions?

[ankiga-MSFT]

We are building an AI assistant using Semantic Kernel. We have written a set of Native Functions in a Plugin, and we are using the AutoInvokeKernelFunctions for AI assistant to identify the right function(s) and chain the calls to finally produce the result.

We are observing that while doing so, Semantic Kernel is giving very slow performance.

To rule out issues with the network calls inside our functions & client logic, we tried to fiddle with the example as shown here

Tried a sample question - "Take the square root of 17 and 11 and add the results. Finally, round off the value to 1 decimal place."

This is resulting in 4 function calls (Sqrt + Sqrt + Add + Round) which is perfectly fine. But in order to do the entire operation, it took a whopping 48 seconds. The model gpt-4 was used for this Test deployed in an Azure Open AI instance.

Adding a Stepwise Planner slightly improved the performance (35 seconds) but still it is slow, considering the fact that hitting the same query using the prompt (without any Math Plugins) yields the result in 6 seconds.

Any idea on what is going wrong here, or how we can improve performance for similar use cases involving multiple function calls. Also, how do we identify how many calls were made to Azure Open AI instance for solving a query.

[markwallace-microsoft]

@ankiga-MSFT
We are adding new filters so that you can monitor the behaviour when AutoInvokeKernelFunctions is being used. Here is the PR #5936. We should have this code merged and in a release in the next week or so. You can then use the filters to monitor what is going on in the function calling loop with the LLM.