Maximum five plugin calls when used with reasoning models

[tosokr]

Hi,

I am using the Semantic Kernel Python 1.32.2 in combination with ChainLit 2.5.5 and the O4-mini model.

I am creating an agent (tried with both ChatCompletionAgent and AzureResponsesAgent) and attaching four plugins (two MCP servers, one open API, and one Azure Agent) to the kernel.

If I use agent.get_response(), it works, but the plugins are called a maximum of five times. With agent.invoke_stream(), it just hangs out after the fifth call.

Is there a limit to using the semantic kernel with reasoning models? Any idea what is wrong?
Thanks

[TaoChenOSU]

Hi @tosokr,

Thank you for your question!

Yes, by default, Semantic Kernel limits the number of auto function invocation attempts to 5 times. This is to prevent models from getting in infinite loops. However, if you are confident about your functions (i.e. clear descriptions and low costs), you can increase the limit. You can do so by providing a custom FunctionChoiceBehavior object:

from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior

agent = ChatCompletionAgent(
    service=AzureChatCompletion(),
    name="...",
    instructions="...",
    plugins=[...],
    function_choice_behavior=FunctionChoiceBehavior.Auto(maximum_auto_invoke_attempts=25),  # Custom FunctionChoiceBehavior
)

Regarding to the hanging issue, is it possible for you to know where it hangs? And is it on the ChantCompletionAgent or the AzureResponsesAgent?

[tosokr]

Hi @TaoChenOSU

Thanks for the provided answer.

About the hanging issue, I am using ChatCompletionAgent. There is no error reported, so I don't know what the problem is.

[TaoChenOSU]

If you can provide the minimum code for us to reproduce the hanging issue, we can assist you further.

[moonbox3]

Hi @tosokr, you could also give the AzureResponseAgent a shot. It may handle the reasoning model better? But as @TaoChenOSU mentioned, it'd be great to get some code to try and repro the issue. There shouldn't be any difference between streaming and non-streaming.

[tosokr]

I also tried with AzureResponseAgent, but I observed the same behavior.

Here is a code to reproduce (I use the Azure o4-mini model):

import os
from dotenv import load_dotenv
import chainlit as cl
import semantic_kernel as sk
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureChatPromptExecutionSettings
from semantic_kernel.agents import ChatCompletionAgent, ChatHistoryAgentThread
from semantic_kernel.connectors.mcp import MCPStdioPlugin
from semantic_kernel.connectors.ai import FunctionChoiceBehavior
from semantic_kernel.functions.kernel_arguments import KernelArguments
import logging
from semantic_kernel.utils.logging import setup_logging

agent = None

@cl.on_app_startup
async def on_app_startup():

setup_logging()
logging.getLogger("kernel").setLevel(logging.DEBUG)    
  
env_github = {        
    "GITHUB_PERSONAL_ACCESS_TOKEN": os.getenv("GITHUB_PAT"),
    "GITHUB_TOOLSETS": "repos"
}    

mcp_server_github = MCPStdioPlugin(
        name="github",
        description="MCP server for Github",
        command="bin/github-mcp-server-0.4.0",
        args=["stdio"],
        env=env_github
)      

await mcp_server_github.connect()
                

 # Setup Semantic Kernel
kernel = sk.Kernel()    

# ensure that the reasoning_model is always a boolean
  
ai_service = AzureChatCompletion(service_id="reasoning", instruction_role="developer")   
request_settings = AzureChatPromptExecutionSettings(
    service_id="reasoning",
    max_completion_tokens=5000,
    reasoning_effort="medium",
    function_choice_behavior=FunctionChoiceBehavior.Auto(maximum_auto_invoke_attempts=2)
)         
kernel.add_service(ai_service)    
    
   
# Instantiate and add the Chainlit filter to the kernel
# This will automatically capture function calls as Steps
sk_filter = cl.SemanticKernelFilter(kernel=kernel)

global agent
agent = ChatCompletionAgent(
    kernel=kernel,     
    name="Host",
    description="My Agent",                
    instructions="You are a helpful assistant.",
    arguments=KernelArguments(settings=request_settings),
    plugins=[mcp_server_github],             
)    

@cl.on_chat_start
async def on_chat_start():

thread = ChatHistoryAgentThread()    

cl.user_session.set("thread", thread)

@cl.on_message
async def on_message(message: cl.Message):
global agent
thread = cl.user_session.get("thread")

answer = cl.Message(content="")
    
async for response in agent.invoke_stream(messages=message.content, thread=thread):

    if response.content:
        await answer.stream_token(str(response.content))

    thread = response.thread
    cl.user_session.set("thread", thread)

# Send the final message
await answer.send()