Skip to content

Commit c084b06

Browse files
moonbox3moonbox3
authored
Python: Support structured outputs with Azure AI inference chat completion. (#12183)
### Motivation and Context The Azure AI inference SDK released support for structured outputs for chat completions (for models that support it). We had not brought this into SK. This PR adds functionality and the ability to run the structured output samples that exist today. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description Adds functionality to handle structured outputs for the Azure AI Inference Chat Completion calls. - Fixes a bug within AzureAIInferenceChatCompletion where the wrong isinstance check type is used. This prevented adding tool calls during streaming invocation. Also properly add the `ai_model_id` from the connector during message construction during streaming invocation, - Adds unit tests for AzureAIInferenceChatCompletion structured outputs - Closes #11952 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄
1 parent 318e092 commit c084b06

File tree

6 files changed

+354
-172
lines changed

6 files changed

+354
-172
lines changed

‎python/samples/concepts/structured_outputs/json_structured_outputs.py

Lines changed: 77 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,107 +1,95 @@
11
# Copyright (c) Microsoft. All rights reserved.
22

33
import asyncio
4+
import json
45

6+
from pydantic import BaseModel, ConfigDict
7+
8+
from samples.concepts.setup.chat_completion_services import Services, get_chat_completion_service_and_request_settings
59
from semantic_kernel import Kernel
6-
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior
7-
from semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion import AzureChatCompletion
8-
from semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion import OpenAIChatCompletion
9-
from semantic_kernel.contents import ChatHistory
10-
from semantic_kernel.contents.streaming_chat_message_content import StreamingChatMessageContent
11-
12-
###################################################################
13-
# The following sample demonstrates how to create a chat #
14-
# completion call that assists users in solving math problems. #
15-
# The bot guides the user step-by-step through the solution #
16-
# process using a structured output format based on either a #
17-
# Pydantic model or a non-Pydantic model. #
18-
###################################################################
19-
20-
21-
###################################################################
22-
# NOTE: If using Azure OpenAI the the following is required:
23-
# - access to gpt-4o-2024-08-06
24-
# - the 2024-08-01-preview API version
25-
# - if using a token instead of an API KEY, you must have the
26-
# `Cognitive Services OpenAI Contributor` role assigned to your
27-
# Azure AD user.
28-
# - flip the `use_azure_openai` flag to `True`
29-
###################################################################
30-
use_azure_openai = False
10+
from semantic_kernel.connectors.ai import FunctionChoiceBehavior
11+
from semantic_kernel.contents import ChatHistory, StreamingChatMessageContent
12+
13+
"""
14+
The following sample demonstrates how to create a chat
15+
completion call that assists users in solving math problems.
16+
The bot guides the user step-by-step through the solution
17+
process using a structured output format based on either a
18+
Pydantic model or a non-Pydantic model.
19+
20+
21+
NOTE: If using Azure OpenAI the the following is required:
22+
- access to gpt-4o-2024-08-06
23+
- the 2024-08-01-preview API version
24+
- if using a token instead of an API KEY, you must have the
25+
`Cognitive Services OpenAI Contributor` role assigned to your
26+
Azure AD user.
27+
- flip the `use_azure_openai` flag to `True`
28+
"""
3129

3230
system_message = """
3331
You are a helpful math tutor. Guide the user through the solution step by step.
3432
"""
3533

34+
"""
35+
Define the Pydantic model that represents the
36+
structured output from the OpenAI service. This model will be
37+
used to parse the structured output from the OpenAI service,
38+
and ensure that the model correctly outputs the schema based
39+
on the Pydantic model.
40+
"""
3641

37-
###################################################################
38-
# OPTION 1: Define the Pydantic model that represents the
39-
# structured output from the OpenAI service. This model will be
40-
# used to parse the structured output from the OpenAI service,
41-
# and ensure that the model correctly outputs the schema based
42-
# on the Pydantic model.
43-
from semantic_kernel.kernel_pydantic import KernelBaseModel # noqa: E402
42+
# Note: The `extra=forbid` means to forbid extra fields during model initialization
43+
# It is required to ensure that the model is strict and does not
44+
# accept any extra fields that are not defined in the model.
4445

4546

46-
class Step(KernelBaseModel):
47+
class Step(BaseModel):
48+
model_config = ConfigDict(extra="forbid")
4749
explanation: str
4850
output: str
4951

5052

51-
class Reasoning(KernelBaseModel):
53+
class Reasoning(BaseModel):
54+
model_config = ConfigDict(extra="forbid")
5255
steps: list[Step]
5356
final_answer: str
5457

5558

56-
###################################################################
57-
58-
59-
# OPTION 2: Define a non-Pydantic model that should represent the
60-
# structured output from the OpenAI service. This model will be
61-
# converted to the proper JSON Schema and sent to the LLM.
62-
# Uncomment the follow lines and comment out the Pydantic model
63-
# above to use this option.
64-
# class Step:
65-
# explanation: str
66-
# output: str
67-
68-
69-
# class Reasoning:
70-
# steps: list[Step]
71-
# final_answer: str
72-
73-
74-
###################################################################
75-
7659
kernel = Kernel()
7760

78-
service_id = "structured-output"
79-
if use_azure_openai:
80-
chat_service = AzureChatCompletion(
81-
service_id=service_id,
82-
)
83-
else:
84-
chat_service = OpenAIChatCompletion(
85-
service_id=service_id,
86-
)
87-
kernel.add_service(chat_service)
88-
89-
req_settings = kernel.get_prompt_execution_settings_from_service_id(service_id=service_id)
90-
req_settings.max_tokens = 2000
91-
req_settings.temperature = 0.7
92-
req_settings.top_p = 0.8
93-
req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"excluded_plugins": ["chat"]})
61+
# You can select from the following chat completion services:
62+
# Note: the model must allow for structured outputs.
63+
# - Services.OPENAI
64+
# - Services.AZURE_OPENAI
65+
# - Services.AZURE_AI_INFERENCE
66+
# - Services.ANTHROPIC
67+
# - Services.BEDROCK
68+
# - Services.GOOGLE_AI
69+
# - Services.MISTRAL_AI
70+
# - Services.OLLAMA
71+
# - Services.ONNX
72+
# - Services.VERTEX_AI
73+
# - Services.DEEPSEEK
74+
# Please make sure you have configured your environment correctly for the selected chat completion service.
75+
chat_completion_service, request_settings = get_chat_completion_service_and_request_settings(Services.AZURE_OPENAI)
76+
kernel.add_service(chat_completion_service)
77+
78+
request_settings.max_tokens = 2000
79+
request_settings.temperature = 0.7
80+
request_settings.top_p = 0.8
81+
request_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"excluded_plugins": ["chat"]})
9482

9583
# NOTE: This is the key setting in this example that tells the OpenAI service
9684
# to return structured output based on the Pydantic model Reasoning.
97-
req_settings.response_format = Reasoning
85+
request_settings.response_format = Reasoning
9886

9987

10088
chat_function = kernel.add_function(
10189
prompt=system_message + """{{$chat_history}}""",
10290
function_name="chat",
10391
plugin_name="chat",
104-
prompt_execution_settings=req_settings,
92+
prompt_execution_settings=request_settings,
10593
)
10694

10795
history = ChatHistory()
@@ -127,9 +115,23 @@ async def main():
127115
chat_function,
128116
chat_history=history,
129117
)
130-
reasoned_result = Reasoning.model_validate_json(result.value[0].content)
131-
print(f"Mosscap:> {reasoned_result}")
132-
history.add_assistant_message(str(result))
118+
reasoned_result = Reasoning.model_validate(json.loads(result.value[0].content))
119+
print(f"{reasoned_result.model_dump_json(indent=4)}")
120+
history.add_assistant_message(str(result))
121+
122+
"""
123+
Sample Output:
124+
125+
{
126+
"steps": [
127+
{
128+
"explanation": "User requested the current weather condition in Paris, so I utilized the 'weather-get_weather_for_city' function to retrieve the data.",
129+
"output": "The weather in Paris is 60 degrees Fahrenheit and rainy."
130+
}
131+
],
132+
"final_answer": "The current weather in Paris is 60 degrees Fahrenheit and rainy."
133+
}
134+
""" # noqa: E501
133135

134136

135137
if __name__ == "__main__":

0 commit comments

Comments
 (0)