Python: Bug: Python - Unable to call plugin during process step

[Athosone]

Describe the bug
I am writing a process in python.

When calling the ai service, If I want to use function call then I got this error:

[2025-05-14 23:40:34 - semantic_kernel.kernel:206 - ERROR] Something went wrong in function invocation. During function invocation: 'GenerateDocumentationStep-GENERATE_DOCUMENTATION'. Error description: '("<class 'semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion.AzureChatCompletion'> service failed to complete the prompt", TypeError('Object of type _PydanticGeneralMetadata is not JSON serializable'))'

To Reproduce
Steps to reproduce the behavior:

Create a process like this:

main.py:

async def main():
    async with (
        MCPStdioPlugin(
            name="filesystem",
            description="Plugin to access and interact with the filesystem",
            command="npx",
            args=[
                "-y",
                "@modelcontextprotocol/server-filesystem",
                "/home/athosone/git/",
            ],
        ) as filesystem_plugin,
    ):
        model = load_chat_model()
        process = create_process()
        kernel = Kernel()
        kernel.add_service(model)
        kernel.add_plugin(filesystem_plugin)
        setup_logging()
        process_context = await start(
            process=process,
            kernel=kernel,
            initial_event=KernelProcessEvent(
                id="Start", data="/blabla"
            ),
        )
        _ = await process_context.get_state()

process:

def create_process() -> KernelProcess:
    process_builder = ProcessBuilder(name="DocumentationGeneration")

    info_gathering_step = process_builder.add_step(GatherUsecasesStep)
    docs_generation_step = process_builder.add_step(GenerateDocumentationStep)
    docs_publish_step = process_builder.add_step(PublishDocumentationStep)

    process_builder.on_input_event("Start").send_event_to(target=info_gathering_step)

    info_gathering_step.on_function_result(
        function_name=GatherUsecasesStep.Functions.GATHER_USECASES.name
    ).send_event_to(
        target=docs_generation_step,
        function_name=GenerateDocumentationStep.Functions.GENERATE_DOCUMENTATION.name,
        parameter_name="usecases",
    )

    docs_generation_step.on_event(
        GenerateDocumentationStep.OutputEvents.DOCUMENTATION_GENERATED
    ).send_event_to(target=docs_publish_step)

    kernel_process = process_builder.build()
    return kernel_process

the step that fails:

    @kernel_function(name=Functions.GENERATE_DOCUMENTATION.name)
    async def generate_documentation(
        self,
        context: KernelProcessStepContext,
        kernel: Kernel,
        usecases: dict[str, str],
    ) -> None:
        print(
            f"{GenerateDocumentationStep.__name__}\n\t Generating documentation for provided usecases"
        )

        completion_client, settings = kernel.select_ai_service(type=AzureChatCompletion)

        assert isinstance(completion_client, ChatCompletionClientBase)  # nosec
        assert isinstance(settings, AzureChatPromptExecutionSettings)

        settings.function_choice_behavior = FunctionChoiceBehavior.Auto() # If I remove this line then I can sucessfully call the llm (but without the functions...)

        usecase_chat = ChatHistory(system_message=self.system_prompt)
        usecase_chat.add_user_message(
            """
            Hello
            """
        )
        r = await completion_client.get_chat_message_content(
            chat_history=usecase_chat,
            settings=settings,
            kernel=kernel,
        )
        print(r)

Expected behavior
I can call the completion client with function calling enabled.

Platform

• Language: Python 3.12.8
• AI model: AzureOpenAI gpt-4o
• OS: Linux (wsl2)

Additional Context

If I call it from main it works with the function calling.
It looks like it only failed in the context of a process step.

Complete logs:

Secure MCP Filesystem Server running on stdio
Allowed directories: [ '/home/athosone/git/' ]
Gathering usecases in /home/athosone/git/...
GenerateDocumentationStep                                                                                                                                                                                                                                                                                                            Generating documentation for provided usecases
[2025-05-14 23:40:34 - semantic_kernel.functions.kernel_function:48 - ERROR] Function failed. Error: ("<class 'semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion.AzureChatCompletion'> service failed to complete the prompt", TypeError('Object of type _PydanticGeneralMetadata is not JSON serializable'))
[2025-05-14 23:40:34 - semantic_kernel.kernel:206 - ERROR] Something went wrong in function invocation. During function invocation: 'GenerateDocumentationStep-GENERATE_DOCUMENTATION'. Error description: '("<class 'semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion.AzureChatCompletion'> service failed to complete the prompt", TypeError('Object of type _PydanticGeneralMetadata is not JSON serializable'))'
[2025-05-14 23:40:34 - semantic_kernel.processes.local_runtime.local_step:166 - ERROR] Error in Step GenerateDocumentationStep: Error occurred while invoking function: 'GenerateDocumentationStep-GENERATE_DOCUMENTATION'

[moonbox3]

Thanks for filing, @Athosone. We'll have a look.

[moonbox3]

Hi @Athosone

We have some improvements we can make around making sure we don't try to include non-JSON serializable kernel function arguments while communicating with the model, especially for "reserved" arguments like the kernel.

Based on the sample code you provided, I was able to repro the issue -- it's due to trying to send the kernel JSON schema, which is not serializable in its exact state.

The following should unblock you, update your kernel kernel function arguments to:

# Imports at top of file
from typing import Annotated
...

kernel: Annotated[Kernel | None, "The kernel", {"include_in_function_choices": False}] = None,

As I show here:

from typing import Annotated

from semantic_kernel.connectors.ai import FunctionChoiceBehavior
from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings
from semantic_kernel.connectors.mcp import MCPStdioPlugin
from semantic_kernel.contents import ChatHistory
from semantic_kernel.functions import kernel_function
from semantic_kernel.kernel import Kernel
from semantic_kernel.processes import ProcessBuilder
from semantic_kernel.processes.kernel_process import (
    KernelProcess,
    KernelProcessEvent,
    KernelProcessStep,
    KernelProcessStepContext,
)
from semantic_kernel.processes.local_runtime.local_kernel_process import start


def load_chat_model():
    return OpenAIChatCompletion()


class GatherUsecasesStep(KernelProcessStep):
    class Functions:
        GATHER_USECASES = "GatherUsecases"

    @kernel_function(name=Functions.GATHER_USECASES)
    async def gather_usecases(
        self,
        context: KernelProcessStepContext,
        usecase: str,
        kernel: Annotated[Kernel | None, "The kernel", {"include_in_function_choices": False}] = None,
    ) -> dict[str, str]:
        print(f"{GatherUsecasesStep.__name__}\n\t Gathering usecases for {usecase}")
        return {"usecase": usecase}


class GenerateDocumentationStep(KernelProcessStep):
    class Functions:
        GENERATE_DOCUMENTATION = "GenerateDocumentation"

    @kernel_function(name=Functions.GENERATE_DOCUMENTATION)
    async def generate_documentation(
        self,
        context: KernelProcessStepContext,
        usecases: dict[str, str],
        kernel: Annotated[Kernel | None, "The kernel", {"include_in_function_choices": False}] = None,
    ) -> None:
        print(f"{GenerateDocumentationStep.__name__}\n\t Generating documentation for provided usecases")

        completion_client, settings = kernel.select_ai_service(type=OpenAIChatCompletion)

        assert isinstance(completion_client, ChatCompletionClientBase)  # nosec
        assert isinstance(settings, OpenAIChatPromptExecutionSettings)

        settings.function_choice_behavior = (
            FunctionChoiceBehavior.Auto()
        )  # If I remove this line then I can sucessfully call the llm (but without the functions...)

        usecase_chat = ChatHistory()
        usecase_chat.add_user_message(
            """
            What files are available in the current directory?
            """
        )
        r = await completion_client.get_chat_message_content(
            chat_history=usecase_chat,
            settings=settings,
            kernel=kernel,
        )
        print(r)


def create_process() -> KernelProcess:
    process_builder = ProcessBuilder(name="DocumentationGeneration")

    info_gathering_step = process_builder.add_step(GatherUsecasesStep)
    docs_generation_step = process_builder.add_step(GenerateDocumentationStep)

    process_builder.on_input_event("Start").send_event_to(target=info_gathering_step, parameter_name="usecase")

    info_gathering_step.on_function_result(function_name=GatherUsecasesStep.Functions.GATHER_USECASES).send_event_to(
        target=docs_generation_step,
        function_name=GenerateDocumentationStep.Functions.GENERATE_DOCUMENTATION,
        parameter_name="usecases",
    )

    return process_builder.build()


async def main():
    async with (
        MCPStdioPlugin(
            name="filesystem",
            description="Plugin to access and interact with the filesystem",
            command="npx",
            args=[
                "-y",
                "@modelcontextprotocol/server-filesystem",
                "<absolute-path>",
            ],
        ) as filesystem_plugin,
    ):
        model = load_chat_model()
        process = create_process()
        kernel = Kernel()
        kernel.add_service(model)
        kernel.add_plugin(filesystem_plugin)
        async with await start(
            process=process,
            kernel=kernel,
            initial_event=KernelProcessEvent(id="Start", data="/blabla"),
        ) as process_context:
            process_state = await process_context.get_state()
            print(f"Process state: {process_state}")


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

I see this response:

GatherUsecasesStep
         Gathering usecases for /blabla
GenerateDocumentationStep
         Generating documentation for provided usecases
The available files and directories in the current directory are:

Files:
- TOC.yml

Directories:
- Frameworks
- api-docs
- breadcrumb
- concepts
- get-started
- media
- mermaid
- overview
- support

Let me know if you want details about any specific file or folder!

One note: you can simplify the interaction a lot with the model by introducing a ChatCompletionAgent if you wish:

class GenerateDocumentationStep(KernelProcessStep):
    class Functions:
        GENERATE_DOCUMENTATION = "GenerateDocumentation"

    @kernel_function(name=Functions.GENERATE_DOCUMENTATION)
    async def generate_documentation(
        self,
        context: KernelProcessStepContext,
        usecases: dict[str, str],
        kernel: Annotated[Kernel | None, "The kernel", {"include_in_function_choices": False}] = None,
    ) -> None:
        print(f"{GenerateDocumentationStep.__name__}\n\t Generating documentation for provided usecases")

        # Function call behavior is enabled to Auto by default, can always change by passing in
        # the `function_choice_behavior` kwarg
        agent = ChatCompletionAgent(
            service=OpenAIChatCompletion(),  # Update with your AI service
            kernel=kernel,
            instructions="Answer questions about my file system.",  # Update with your instructions
        )

        response = await agent.get_response(messages="What files are available in the current directory?")
        print(response)

[alfredzouang]

Hi @moonbox3 ,

Thank you for your reply. I have met the exact issue. However I tried your method kernel: Annotated[Kernel | None, "The kernel", {"include_in_function_choices": False}] = None and the error persists.

Here is my code:

@kernel_function(name=Functions.CREATE_TASK.value)
    async def create_tasks(self, context: KernelProcessStepContext,  project: Project, kernel: Annotated[Kernel | None, "The kernel", {"include_in_function_choices": False}] = None) -> None:

        logger.info(f"Creating tasks for project: {project.description}")

        self.state.project_infos.project = project
        self.state.chat_history.add_user_message(json.dumps(project.model_dump()))

        chat_service, settings = kernel.select_ai_service(type=ChatCompletionClientBase)
        assert isinstance(chat_service, ChatCompletionClientBase)
        assert isinstance(settings, AzureChatPromptExecutionSettings)
        settings.response_format = CreateProjectTaskResponse
        settings.function_choice_behavior = FunctionChoiceBehavior.Auto()
        settings.temperature = 0.0
        settings.max_tokens = 6000

        agent = ChatCompletionAgent(
            # service=chat_service,
            kernel=kernel,
            name = "CreateProjectTaskAgent",
            instructions= self.system_prompt.format(
                input_format=Project.model_json_schema(),
                output_format=CreateProjectTaskResponse.model_json_schema()
            ),
            arguments=KernelArguments(settings=settings)
        )
        thread = ChatHistoryAgentThread(chat_history=self.state.chat_history)

        response = await agent.get_response(thread=thread)
        self.state.chat_history.add_assistant_message(response.message.content)

        formatted_response: CreateProjectTaskResponse = CreateProjectTaskResponse.model_validate_json(response.message.content)
        task_list = formatted_response.task_list
        self.state.project_infos.task_list = task_list

        await context.emit_event(process_event=self.OutputEvents.TASK_CREATED, data=formatted_response.model_dump())

And still I got the error:

[2025-05-15 14:57:46 - semantic_kernel.functions.kernel_function:48 - ERROR] Function failed.      kernel_function_log_messages.py:48
                             Error: ("<class                                                                                                                      
                             'semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion.AzureChatCompletion'>                                          
                             service failed to complete the prompt", TypeError('Object of type _PydanticGeneralMetadata is not                                    
                             JSON serializable'))  

Did I miss anything? Thank you for your kindness.

[Athosone]

Hello !
Thanks for taking the time to answer,

Using the ChatCompletionAgent yield the same error, however your proposal to add {"include_in_function_choices": False} make it work.

Thanks a lot for the suggestion I will indeed use the ChatCompletionAgent along side the annotation, it make things more concise.

[moonbox3]

Hi @alfredzouang, is your Project object serializable? Another option to handle this (while I investigate further), is to use the FunctionChoiceBehavior filters:

settings.function_choice_behavior=FunctionChoiceBehavior.Auto(filters={"included_plugins": ["<name-of-plugin>"]}),

# or

settings.function_choice_behavior=FunctionChoiceBehavior.Auto(filters={"excluded_plugins": ["<name-of-plugin>"]}),

The steps from the process are getting included with the JSON schema that is sent to the model during agent invocation. All functions currently registered on the kernel are having their schemas sent over as tools.

As I said, I will look at how to better handle this.

[moonbox3]

Thanks for reporting back, @Athosone. We'll still look for a more elegant solution here (longer-term) but to unblock you, glad you are able to add the Annotation metadata.