message="Unknown parameter: 'session.output_modalities' While using gpt realtime api

[ANYMS-A]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

Hey, I am using openai==1.107.3 . Currently following this example script to build realtime api chatbot https://github.com/openai/openai-python/blob/main/examples/realtime/azure_realtime.py

In the example code line#34 and line#36 there are two param will lead to a ErrorEvent
"output_modalities": ["text"],
"type": "realtime",

If set any of those two param in the session param, I will get:

RealtimeErrorEvent(error=RealtimeError(message="Unknown parameter: 'session.output_modalities'.", type='invalid_request_error', code='unknown_parameter', event_id=None, param='session.output_modalities'), event_id='event_CGz0MgUoIe6epd4eHp4C2', type='error')

BTW, after checking the RealtimeSessionCreateRequestParam implementation in .venv\Lib\site-packages\openai\types\realtime\realtime_session_create_request_param.py. I thought the correct way to set the voice is:

{
    "audio": {"output": "marin"}
}

But this will lead to the same RealtimeErrorEvent

RealtimeErrorEvent(error=RealtimeError(message="Unknown parameter: 'session.audio'.", type='invalid_request_error', code='unknown_parameter', event_id=None, param='session.audio'), event_id='event_CGz9MD6lksIcT4iljdr1r', type='error')

If I set voice directly, it works
{
"voice": "marin"
}

To Reproduce

using openai openai==1.107.3
create your async azure client and run the code below

Code snippets

import os
import asyncio


from loguru import logger
from openai import AsyncAzureOpenAI
from azure_service_man.azure_service import AzureService


async def main() -> None:
    """The following example demonstrates how to configure Azure OpenAI to use the Realtime API.
    For an audio example, see push_to_talk_app.py and update the client and model parameter accordingly.

    When prompted for user input, type a message and hit enter to send it to the model.
    Enter "q" to quit the conversation.
    """

    az_service = AzureService(api_version="2024-10-01-preview")
    client: AsyncAzureOpenAI = az_service.client
    async with client.realtime.connect(
        model="GenAI_SG_AT_NonProdUSE201_gpt-realtime",  # deployment name for your model
    ) as connection:
        await connection.session.update(
            session={
                "output_modalities": ["text"],
                "model": "gpt-realtime",
                "type": "realtime",
                "instructions": "Speak with Singapore accent.",
                "audio": {"output": {"voice": "marin"}},
            }
        )
        async for event in connection:
            logger.info(event)

        while True:
            user_input = input("Enter a message: ")
            if user_input == "q":
                break

            await connection.conversation.item.create(
                item={
                    "type": "message",
                    "role": "user",
                    "content": [{"type": "input_text", "text": user_input}],
                }
            )
            await connection.response.create()
            async for event in connection:
                if event.type == "response.output_text.delta":
                    print(event.delta, flush=True, end="")
                elif event.type == "response.output_text.done":
                    print()
                elif event.type == "response.done":
                    break

    await credential.close()


asyncio.run(main())

OS

Windows

Python version

3.13.2

Library version

v1.107.3

[tinalenguyen]

Hi, Azure currently doesn't support the general access version so you'd have to work according to the beta API. The error log is in respect to the Session shape; see here for the Azure Realtime docs.

Related issue: livekit/agents#3489

If you're interested, here's a quickstart guide for using Azure OpenAI Realtime API as a chatbot/agent. Hopefully this helps!

[eric-fb]

Hitting same issue, is this blocking on Azure to update their development?

[abi-jey]

You can probably use the custom url to force api to use v1 same is discussed here openai/openai-agents-python#1748

[JiwaniZakir]

The message="Unknown parameter: 'session.output_modalities' While using gpt realtime api issue seems straightforward enough. I'll work through it and push a fix.

[khalidsaidi]

A2ABench has an accepted answer for this imported thread.

• Thread: https://a2abench-api.web.app/q/cmmpna6v000hk14019d9bvn7m
• Accepted at: 2026-03-25T17:17:03.202Z
• Accepted answer agent: ai village
• Answer preview: "This looks real, but it does not look like a low-level openai-python transport bug. It looks like an Azure compatibility / stale example mismatch in the Python repo’s Azure Realtime example. Why: The error is coming back from the server as unknown_parameter after the websocket co"