Python: Bug: Validation and Plugin Errors When Using AzureAISearchDataSource with Agent

[MohanadElMetwally]

I am trying to connect the ChatCompletionAgent to an Azure AI Search instance by adding it to extra_body in AzureChatPromptExecutionSettings:

settings = AzureChatPromptExecutionSettings(
    extra_body=ExtraBody(
        data_sources=[AzureAISearchDataSource(parameters=ds_params)]
    )
)

However, when I tried to invoke and stream the agent, I received the following validation error:

ServiceResponseException: ("<class 'semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion.AzureChatCompletion'> service failed to complete the prompt", BadRequestError("Error code: 400 - {'error': {'requestid': '79734843-6a5a-4c03-8378-930f991070b2', 'code': 400, 'message': 'Validation error at #/messages/6/name: Extra inputs are not permitted\nValidation error at #/stream_options: Extra inputs are not permitted'}}"))

To work around this, I tried monkey-patching by modifying _send_completion_request in open_ai_handler.py as follows:

settings_dict = settings.prepare_settings_dict()
extra = getattr(settings, "extra_body", None)
has_search = extra and getattr(extra, "data_sources", None)

if settings_dict.get("stream") and has_search:
    settings_dict.pop("stream_options", None)

if has_search and "messages" in settings_dict:
    for msg in settings_dict["messages"]:
        msg.pop("name", None)

clean_msgs = []
for msg in settings_dict["messages"]:
    if isinstance(msg.get("content", None), str):
        clean_msgs.append(msg)

settings_dict["messages"] = clean_msgs

This workaround allowed the agent to stream responses using the documents retrieved from the data source. However, at the end, it tried to call a plugin and raised the following error:

ERROR:semantic_kernel.kernel:The function Azure-OnYourData is not part of the provided functions: Plugin 'Azure' not found.
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/semantic_kernel/kernel.py", line 357, in invoke_function_call
    function_to_call = self.get_function(function_call.plugin_name, function_call.function_name)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/semantic_kernel/functions/kernel_function_extension.py", line 294, in get_function
    raise KernelPluginNotFoundError(f"Plugin '{plugin_name}' not found")
semantic_kernel.exceptions.kernel_exceptions.KernelPluginNotFoundError: Plugin 'Azure' not found

Here is the output of what happened when I used agent.invoke_stream() before I interrupted it:

Object-Oriented Programming (OOP) is a programming paradigm that organizes software design around data, or objects, rather than functions and logic. It is based on the concept of "objects," which can contain data in the form of fields (often called attributes) and code in the form of procedures (often called methods). OOP emphasizes the following key principles:

1. **Encapsulation**: Bundling the data and methods that operate on the data into a single unit, or class, and restricting access to some of the object's components for security and simplicity [doc4].
2. **Inheritance**: Allowing a new class to inherit the properties and methods of an existing class, promoting code reuse and reducing redundancy .
3. **Polymorphism**: Enabling objects to be treated as instances of their parent class, allowing one interface to be used for a general class of actions .
4. **Dynamic Binding**: Linking a procedure call to the code to be executed at runtime, rather than compile time, for flexibility [doc5].
5. **Message Passing**: Objects communicate with each other by sending and receiving information, invoking methods to perform tasks [doc5].

OOP provides several benefits, including reusabilityERROR:semantic_kernel.kernel:The function `Azure-OnYourData` is not part of the provided functions: Plugin 'Azure' not found.
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/semantic_kernel/kernel.py", line 357, in invoke_function_call
    function_to_call = self.get_function(function_call.plugin_name, function_call.function_name)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/semantic_kernel/functions/kernel_function_extension.py", line 294, in get_function
    raise KernelPluginNotFoundError(f"Plugin '{plugin_name}' not found")
semantic_kernel.exceptions.kernel_exceptions.KernelPluginNotFoundError: Plugin 'Azure' not found
, data hiding for security, reduced complexity, and easier maintenance and upgrades [doc5]. It is widely used in modern programming languages like C++, Java, and Python.Object-Oriented Programming (OOP) is a programming paradigm based on the concept of "objects," which can contain data in the form of fields (often called attributes or properties) and code in the form of procedures (often called methods). OOP focuses on organizing code into reusable and modular components, making it easier to design, maintain, and scale software systems.

Key principles of OOP include:

1. **Encapsulation**: Bundling data and methods that operate on the data within a single unit (object) and restricting access to some of the object's components.
2. **Inheritance**: Allowing a class to inherit properties and methods from another class, promoting code reuse.
3. **Polymorphism**: Enabling objects to be treated as instances of their parent class, allowing one interface to be used for different underlying forms (e.g., method overriding).
4. **Abstraction**: Hiding complex implementation details and showing only the necessary features of an object.

OOP is widely usedERROR:semantic_kernel.kernel:The function `Azure-OnYourData` is not part of the provided functions: Plugin 'Azure' not found.
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/semantic_kernel/kernel.py", line 357, in invoke_function_call
    function_to_call = self.get_function(function_call.plugin_name, function_call.function_name)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/semantic_kernel/functions/kernel_function_extension.py", line 294, in get_function
    raise KernelPluginNotFoundError(f"Plugin '{plugin_name}' not found")
semantic_kernel.exceptions.kernel_exceptions.KernelPluginNotFoundError: Plugin 'Azure' not found
 in programming languages like Java, Python, C++, and C#.Object-Oriented Programming (OOP) is a programming paradigm that organizes software design around data, or objects, rather than functions and logic. It focuses on creating reusable and maintainable code by structuring programs as a collection of objects, each containing data and methods to manipulate that data [doc1][doc2].

[moonbox3]

Hi @MohanadElMetwally, I honestly don't have a lot of experience with data_sources=[AzureAISearchDataSource(parameters=ds_params)]. Adding @sarah-widder for any comment/help.