Python: Support Declarative Spec for OpenAIAssistantAgent & OpenAIResponsesAgent (#12247)

### Motivation and Context

This PR builds on the initial work/abstractions from #11982. The
`AzureAssistantAgent` / `OpenAIAssistantAgent` as well as the
`AzureResponsesAgent` / `OpenAIResponsesAgent` now support declarative
specs. Several examples on how to declare the spec are shown in the
respective dirs in `samples/concepts/agents`, as well as a simple
getting started example for each agent type.

To further simplify working with these two agents, the
`setup_resources()` methods for each agent type have been marked with a
deprecation tag. The new method to use is
`OpenAIAssistantAgent.create_client()` or
`OpenAIResponsesAgent.create_client()`. Further, when one needs to
access the underlying setting's, it's simple to do so with a settings
object and accessing the attribute.

In the previous PR for the AzureAIAgent, it lacked support for
templating - this is included in this current PR.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Bring in support for the declarative single agent spec for these agent
types.
- Add samples
- Add unit tests
- Add support for prompt templates for all current agent types
- Closes #11979

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

Author: moonbox3

python/samples/concepts/README.md

@@ -61,12 +61,22 @@
 - [Mixed Chat Reset](./agents/mixed_chat/mixed_chat_reset.py)
 - [Mixed Chat Streaming](./agents/mixed_chat/mixed_chat_streaming.py)
 
-#### [OpenAI Assistant Agent](../../semantic_kernel/agents/open_ai/open_ai_assistant_agent.py)
+#### [OpenAI Assistant Agent](../../semantic_kernel/agents/open_ai/openai_assistant_agent.py)
 
+- [Azure OpenAI Assistant Declarative Code Interpreter](./agents/openai_assistant/azure_openai_assistant_declarative_code_interpreter.py)
+- [Azure OpenAI Assistant Declarative File Search](./agents/openai_assistant/azure_openai_assistant_declarative_file_search.py)
+- [Azure OpenAI Assistant Declarative Function Calling From File](./agents/openai_assistant/azure_openai_assistant_declarative_function_calling_from_file.py)
+- [Azure OpenAI Assistant Declarative Templating](./agents/openai_assistant/azure_openai_assistant_declarative_templating.py)
+- [Azure OpenAI Assistant Declarative With Existing Agent ID](./agents/openai_assistant/azure_openai_assistant_declarative_with_existing_agent_id.py)
 - [OpenAI Assistant Auto Function Invocation Filter Streaming](./agents/openai_assistant/openai_assistant_auto_func_invocation_filter_streaming.py)
 - [OpenAI Assistant Auto Function Invocation Filter](./agents/openai_assistant/openai_assistant_auto_func_invocation_filter.py)
 - [OpenAI Assistant Chart Maker Streaming](./agents/openai_assistant/openai_assistant_chart_maker_streaming.py)
 - [OpenAI Assistant Chart Maker](./agents/openai_assistant/openai_assistant_chart_maker.py)
+- [OpenAI Assistant Declarative Code Interpreter](./agents/openai_assistant/openai_assistant_declarative_code_interpreter.py)
+- [OpenAI Assistant Declarative File Search](./agents/openai_assistant/openai_assistant_declarative_file_search.py)
+- [OpenAI Assistant Declarative Function Calling From File](./agents/openai_assistant/openai_assistant_declarative_function_calling_from_file.py)
+- [OpenAI Assistant Declarative Templating](./agents/openai_assistant/openai_assistant_declarative_templating.py)
+- [OpenAI Assistant Declarative With Existing Agent ID](./agents/openai_assistant/openai_assistant_declarative_with_existing_agent_id.py)
 - [OpenAI Assistant File Manipulation Streaming](./agents/openai_assistant/openai_assistant_file_manipulation_streaming.py)
 - [OpenAI Assistant File Manipulation](./agents/openai_assistant/openai_assistant_file_manipulation.py)
 - [OpenAI Assistant Retrieval](./agents/openai_assistant/openai_assistant_retrieval.py)
@@ -79,6 +89,12 @@
 
 #### [OpenAI Responses Agent](../../semantic_kernel/agents/open_ai/openai_responses_agent.py)
 
+- [Azure OpenAI Responses Agent Declarative File Search](./agents/openai_responses/azure_openai_responses_agent_declarative_file_search.py)
+- [Azure OpenAI Responses Agent Declarative Function Calling From File](./agents/openai_responses/azure_openai_responses_agent_declarative_function_calling_from_file.py)
+- [Azure OpenAI Responses Agent Declarative Templating](./agents/openai_responses/azure_openai_responses_agent_declarative_templating.py)
+- [OpenAI Responses Agent Declarative File Search](./agents/openai_responses/openai_responses_agent_declarative_file_search.py)
+- [OpenAI Responses Agent Declarative Function Calling From File](./agents/openai_responses/openai_responses_agent_declarative_function_calling_from_file.py)
+- [OpenAI Responses Agent Declarative Web Search](./agents/openai_responses/openai_responses_agent_declarative_web_search.py)
 - [OpenAI Responses Message Callback Streaming](./agents/openai_responses/responses_agent_message_callback_streaming.py)
 - [OpenAI Responses Message Callback](./agents/openai_responses/responses_agent_message_callback.py)
 - [OpenAI Responses File Search Streaming](./agents/openai_responses/responses_agent_file_search_streaming.py)

python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_azure_ai_search.py

@@ -4,8 +4,7 @@
 
 from azure.identity.aio import DefaultAzureCredential
 
-from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
-from semantic_kernel.agents.agent import AgentRegistry
+from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
 from semantic_kernel.contents.chat_message_content import ChatMessageContent
 from semantic_kernel.contents.function_call_content import FunctionCallContent
 from semantic_kernel.contents.function_result_content import FunctionResultContent

python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_bing_grounding.py

@@ -4,8 +4,7 @@
 
 from azure.identity.aio import DefaultAzureCredential
 
-from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
-from semantic_kernel.agents.agent import AgentRegistry
+from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
 
 """
 The following sample demonstrates how to create an Azure AI agent that answers

python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_code_interpreter.py

@@ -6,8 +6,7 @@
 from azure.ai.agents.models import FilePurpose
 from azure.identity.aio import DefaultAzureCredential
 
-from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
-from semantic_kernel.agents.agent import AgentRegistry
+from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
 
 """
 The following sample demonstrates how to create an Azure AI agent that answers

python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_file_search.py

@@ -6,25 +6,20 @@
 from azure.ai.agents.models import VectorStore
 from azure.identity.aio import DefaultAzureCredential
 
-from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
-from semantic_kernel.agents.agent import AgentRegistry
+from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
 
 """
 The following sample demonstrates how to create an Azure AI agent that answers
-user questions using the file search tool.
-
-The agent is used to answer user questions that require file search to help ground 
-answers from the model.
+user questions using the file search tool from a declarative spec.
 """
 
 # Define the YAML string for the sample
 spec = """
 type: foundry_agent
 name: FileSearchAgent
-description: Agent with code interpreter tool.
+description: Agent with file search tool.
 instructions: >
-  Use the code interpreter tool to answer questions that require code to be generated
-  and executed.
+  Use the file search tool to answer questions from the user.
 model:
   id: ${AzureAI:ChatModelId}
   connection:

python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_openapi.py

@@ -4,8 +4,7 @@
 
 from azure.identity.aio import DefaultAzureCredential
 
-from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings
-from semantic_kernel.agents.agent import AgentRegistry
+from semantic_kernel.agents import AgentRegistry, AzureAIAgent, AzureAIAgentSettings
 
 """
 The following sample demonstrates how to create an Azure AI agent that answers

python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_templating.py

@@ -0,0 +1,72 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+import asyncio
+
+from azure.identity.aio import DefaultAzureCredential
+
+from semantic_kernel.agents import AgentRegistry, AzureAIAgent
+
+"""
+The following sample demonstrates how to create an Azure AI Agent that invokes
+a story generation task using a prompt template and a declarative spec.
+"""
+
+# Define the YAML string for the sample
+spec = """
+type: foundry_agent
+name: StoryAgent
+description: An agent that generates a story about a topic.
+instructions: Tell a story about {{$topic}} that is {{$length}} sentences long.
+model:
+  id: ${AzureAI:ChatModelId}
+  connection:
+    connection_string: ${AzureAI:Endpoint}
+inputs:
+  topic:
+    description: The topic of the story.
+    required: true
+    default: Cats
+  length:
+    description: The number of sentences in the story.
+    required: true
+    default: 2
+outputs:
+  output1:
+    description: The generated story.
+template:
+  format: semantic-kernel
+"""
+
+
+async def main():
+    async with (
+        DefaultAzureCredential() as creds,
+        AzureAIAgent.create_client(credential=creds) as client,
+    ):
+        try:
+            # Create the AzureAI Agent from the YAML spec
+            agent: AzureAIAgent = await AgentRegistry.create_from_yaml(
+                yaml_str=spec,
+                client=client,
+            )
+
+            # Invoke the agent for the specified task
+            async for response in agent.invoke(
+                messages=None,
+            ):
+                print(f"# {response.name}: {response}")
+        finally:
+            # Cleanup: Delete the agent, vector store, and file
+            await client.agents.delete_agent(agent.id)
+
+    """
+    Sample output:
+
+    # StoryAgent: Under the silvery moon, three mischievous cats tiptoed across the rooftop, chasing 
+      shadows and sharing secret whispers. By dawn, they curled up together, purring softly, dreaming 
+      of adventures yet to come.
+    """
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

python/samples/concepts/agents/azure_ai_agent/azure_ai_agent_declarative_with_existing_agent_id.py

@@ -4,8 +4,7 @@
 
 from azure.identity.aio import DefaultAzureCredential
 
-from semantic_kernel.agents import AzureAIAgent
-from semantic_kernel.agents.agent import AgentRegistry
+from semantic_kernel.agents import AgentRegistry, AzureAIAgent
 
 """
 The following sample demonstrates how to create an Azure AI agent based 

python/samples/concepts/agents/mixed_chat/mixed_chat_agents_plugins.py

@@ -4,9 +4,9 @@
 from typing import Annotated
 
 from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
-from semantic_kernel.agents.strategies.termination.termination_strategy import TerminationStrategy
+from semantic_kernel.agents.strategies import TerminationStrategy
 from semantic_kernel.connectors.ai import FunctionChoiceBehavior
-from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
+from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
 from semantic_kernel.contents import AuthorRole
 from semantic_kernel.functions import KernelArguments, kernel_function
 from semantic_kernel.kernel import Kernel
@@ -86,11 +86,11 @@ async def main():
     )
 
     # Create the Assistant Agent using Azure OpenAI resources
-    client, model = AzureAssistantAgent.setup_resources()
+    client = AzureAssistantAgent.create_client()
 
     # Create the assistant definition
     definition = await client.beta.assistants.create(
-        model=model,
+        model=AzureOpenAISettings().chat_deployment_name,
         name=COPYWRITER_NAME,
         instructions=COPYWRITER_INSTRUCTIONS,
     )

python/samples/concepts/agents/mixed_chat/mixed_chat_files.py

@@ -4,7 +4,7 @@
 import os
 
 from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
-from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
+from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
 from semantic_kernel.contents import AnnotationContent, AuthorRole
 from semantic_kernel.kernel import Kernel
 
@@ -31,10 +31,10 @@ async def main():
     )
 
     # Create the client using Azure OpenAI resources and configuration
-    client, model = AzureAssistantAgent.setup_resources()
+    client = AzureAssistantAgent.create_client()
 
     # If desired, create using OpenAI resources
-    # client, model = OpenAIAssistantAgent.setup_resources()
+    # client = OpenAIAssistantAgent.create_client()
 
     # Load the text file as a FileObject
     with open(file_path, "rb") as file:
@@ -45,7 +45,7 @@ async def main():
     )
 
     definition = await client.beta.assistants.create(
-        model=model,
+        model=AzureOpenAISettings().chat_deployment_name,
         instructions="Create charts as requested without explanation.",
         name="ChartMaker",
         tools=code_interpreter_tool,

python/samples/concepts/agents/mixed_chat/mixed_chat_images.py

@@ -3,7 +3,7 @@
 import asyncio
 
 from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
-from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
+from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
 from semantic_kernel.contents import AnnotationContent
 from semantic_kernel.contents.utils.author_role import AuthorRole
 from semantic_kernel.kernel import Kernel
@@ -24,14 +24,14 @@ def _create_kernel_with_chat_completion(service_id: str) -> Kernel:
 
 async def main():
     # Create the client using Azure OpenAI resources and configuration
-    client, model = AzureAssistantAgent.setup_resources()
+    client = AzureAssistantAgent.create_client()
 
     # Get the code interpreter tool and resources
     code_interpreter_tool, code_interpreter_resources = AzureAssistantAgent.configure_code_interpreter_tool()
 
     # Create the assistant definition
     definition = await client.beta.assistants.create(
-        model=model,
+        model=AzureOpenAISettings().chat_deployment_name,
         name="Analyst",
         instructions="Create charts as requested without explanation",
         tools=code_interpreter_tool,

python/samples/concepts/agents/mixed_chat/mixed_chat_reset.py

@@ -4,7 +4,7 @@
 from typing import TYPE_CHECKING
 
 from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
-from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
+from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
 from semantic_kernel.contents import AuthorRole
 from semantic_kernel.kernel import Kernel
 
@@ -41,11 +41,11 @@ async def main():
     # Next, we will create the AzureAssistantAgent
 
     # Create the client using Azure OpenAI resources and configuration
-    client, model = AzureAssistantAgent.setup_resources()
+    client = AzureAssistantAgent.create_client()
 
     # Create the assistant definition
     definition = await client.beta.assistants.create(
-        model=model,
+        model=AzureOpenAISettings().chat_deployment_name,
         name="copywriter",
         instructions="""
             The user may either provide information or query on information previously provided. 

python/samples/concepts/agents/mixed_chat/mixed_chat_streaming.py

@@ -4,7 +4,7 @@
 
 from semantic_kernel.agents import AgentGroupChat, AzureAssistantAgent, ChatCompletionAgent
 from semantic_kernel.agents.strategies import TerminationStrategy
-from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
+from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureOpenAISettings
 from semantic_kernel.contents import AuthorRole
 from semantic_kernel.kernel import Kernel
 
@@ -46,11 +46,11 @@ async def main():
     # Next, we will create the AzureAssistantAgent
 
     # Create the client using Azure OpenAI resources and configuration
-    client, model = AzureAssistantAgent.setup_resources()
+    client = AzureAssistantAgent.create_client()
 
     # Create the assistant definition
     definition = await client.beta.assistants.create(
-        model=model,
+        model=AzureOpenAISettings().chat_deployment_name,
         name="CopyWriter",
         instructions="""
         You are a copywriter with ten years of experience and are known for brevity and a dry humor.

python/samples/concepts/agents/openai_assistant/README.md

@@ -32,11 +32,11 @@ OpenAI Assistant Agents are created in the following way:
 from semantic_kernel.agents import OpenAIAssistantAgent
 
 # Create the client using OpenAI resources and configuration
-client, model = OpenAIAssistantAgent.setup_resources()
+client = OpenAIAssistantAgent.create_client()
 
 # Create the assistant definition
 definition = await client.beta.assistants.create(
-    model=model,
+    model=AzureOpenAISettings().chat_deployment_name
     instructions="<instructions>",
     name="<name>",
 )
@@ -73,11 +73,11 @@ Alternatively, you can pass the `api_version` parameter when creating an `AzureA
 from semantic_kernel.agents import AzureAssistantAgent
 
 # Create the client using Azure OpenAI resources and configuration
-client, model = AzureAssistantAgent.setup_resources()
+client = AzureAssistantAgent.create_client()
 
 # Create the assistant definition
 definition = await client.beta.assistants.create(
-    model=model,
+    model=AzureOpenAISettings().chat_deployment_name
     instructions="<instructions>",
     name="<name>",
 )