Python: Introducing Realtime Clients for OpenAI and Azure OpenAI

[eavanvalkenburg]

Motivation and Context

Implements the OpenAI Realtime API with Semantic Kernel

Description

Adds a RealtimeClientBase
Adds RealtimeEvents
Adds OpenAI and Azure Realtime clients

Closes: #10108

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[markwallace-microsoft]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
semantic_kernel
kernel.py	202	47	77%	143, 154, 158, 308–311, 428, 442–485
semantic_kernel/connectors/ai
chat_completion_client_base.py	124	2	98%	398, 408
function_calling_utils.py	48	2	96%	163, 168
realtime_client_base.py	33	1	97%	114
semantic_kernel/connectors/ai/anthropic/services
anthropic_chat_completion.py	162	8	95%	159, 165, 178, 184, 188, 245–247, 380
semantic_kernel/connectors/ai/azure_ai_inference/services
azure_ai_inference_chat_completion.py	110	6	95%	120–123, 132, 156, 182
semantic_kernel/connectors/ai/bedrock/services
bedrock_chat_completion.py	135	14	90%	115, 137, 162, 166–169, 227, 245–264, 323
semantic_kernel/connectors/ai/google/google_ai/services
google_ai_chat_completion.py	122	6	95%	126, 131, 155, 160, 183, 185
semantic_kernel/connectors/ai/google/vertex_ai/services
vertex_ai_chat_completion.py	117	4	97%	123, 150, 176, 178
semantic_kernel/connectors/ai/ollama/services
ollama_chat_completion.py	137	34	75%	116, 141, 145–146, 156, 169, 186, 206–207, 211, 224–234, 245–247, 258–267, 279, 289–290, 312, 323–324, 350, 359–367
semantic_kernel/connectors/ai/open_ai/services
azure_config_base.py	56	2	96%	95, 97
azure_realtime.py	29	8	72%	83–98, 116
open_ai_realtime.py	441	130	71%	342–344, 507–508, 511–512, 515–519, 522, 530–533, 540, 542, 545–546, 596–599, 608–611, 623–624, 663, 669–677, 687–732, 738–739, 742–743, 747–777, 785–794, 798–817, 874–875, 877, 911, 914, 917–927, 930–936, 949, 1010–1011, 1013
semantic_kernel/connectors/ai/open_ai/settings
azure_open_ai_settings.py	26	4	85%	119–122
semantic_kernel/contents
audio_content.py	21	1	95%	88
binary_content.py	117	13	89%	99, 120, 128, 134, 147, 152–153, 194–199
function_call_content.py	107	3	97%	194, 222–223
TOTAL	19586	2551	87%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
3207	5 💤	0 ❌	0 🔥	1m 42s ⏱️

[Copilot]

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[TaoChenOSU]

Is it possible to add some integration tests?