Skip to content

OpenAIRealtimeWebRTC connection_change Fails to Detect Disconnection After Connection is Established #613

@IM594

Description

@IM594

Please read this first

  • Have you read the docs?: Yes
  • Have you searched for related issues?: Yes

Describe the bug

The connection_change event on OpenAIRealtimeWebRTC is broken for connection loss detection. While this event is available immediately after we instantiate new OpenAIRealtimeWebRTC(), it only provides partial coverage of the connection lifecycle.

It correctly reports transitions before the session is active (e.g., "disconnected"-> "connecting" -> "connected"), but it fails to fire reliably when the established connection is lost (i.e., when transitioning from "connected" back to "disconnected", "failed", or "closed").

This means the agent's connection status gets stuck at "connected" in our application state, even when the underlying WebRTC connection is dead.

To get accurate disconnection events, we are forced to manually listen to the RTCPeerConnection.onconnectionstatechange. This is a cumbersome workaround because:

  1. It defeats the purpose of the SDK's abstraction provided by OpenAIRealtimeWebRTC.
  2. We can only reliably attach this manual monitor after the SDK's (flawed) event signals "connected". This creates a clumsy, nested event system just to track basic connection status, preventing us from setting up comprehensive monitoring immediately after instantiating the WebRTC object.

The SDK's current behavior makes robust error handling impossible without this complicated workaround.


Debug information

  • Agents SDK version: v0.1.11
  • Runtime environment: Node.js v22.15.1 and Chrome 141.0.7390.108 (arm64)

Repro steps

  1. Initialize and connect a RealtimeSession using OpenAIRealtimeWebRTC.
  2. Attach a listener to the SDK event: webRTC.on("connection_change", (state) => { console.log("SDK state:", state); }).
  3. Observe that initial states ("disconnected", "connecting", "connected") are correctly reported.
  4. Implement the workaround: Use the webRTC.on("connection_change") event to set up the secondary RTCPeerConnection.onconnectionstatechange monitor once the state is "connected".
  5. Simulate disconnection:
    • Disable your device's network connection (e.g., toggle Wi-Fi off).
  6. Observe that the SDK's connection_change listener does not fire for the transition back to "disconnected", while the manual RTCPeerConnection monitor does.

Expected behavior

The OpenAIRealtimeWebRTC component's connection_change event should report the entire connection lifecycle, including the essential "connected" -> "disconnected" transition, so developers can rely on the SDK's provided events for complete connection state management immediately after instantiation.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions