Skip to content

Update realtime websocket API#13265

Merged
aibrahim-oai merged 15 commits intomainfrom
realtime-api-migration
Mar 3, 2026
Merged

Update realtime websocket API#13265
aibrahim-oai merged 15 commits intomainfrom
realtime-api-migration

Conversation

@aibrahim-oai
Copy link
Collaborator

  • migrate the realtime websocket transport to the new session and handoff flow
  • make the realtime model configurable in config.toml and use API-key auth for the websocket

aibrahim-oai and others added 2 commits March 2, 2026 11:29
- migrate the realtime websocket transport to the new session and handoff flow
- make the realtime model configurable via config.toml and use API-key auth for the websocket

Co-authored-by: Codex <noreply@openai.com>
- group handoff output sender and active handoff id into a dedicated state object
- centralize handoff output enqueue and active handoff lookup behavior

Co-authored-by: Codex <noreply@openai.com>
@aibrahim-oai
Copy link
Collaborator Author

@codex review this

- log in with an API key before starting realtime in app-server realtime tests
- keep the test flow aligned with the new realtime websocket auth requirement

Co-authored-by: Codex <noreply@openai.com>
Copy link
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 75c4cd839d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

aibrahim-oai and others added 7 commits March 2, 2026 12:03
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Keep handoff targeting active across multiple handoff append sends and clear it when the local task emits TurnComplete instead of on conversation.item.done.

Co-authored-by: Codex <noreply@openai.com>
Add an integration test that proves handoff append targeting persists after conversation.item.done and continues until the delegated turn completes.

Co-authored-by: Codex <noreply@openai.com>
Fix clippy::single_match in realtime handoff event handling.

Co-authored-by: Codex <noreply@openai.com>
@aibrahim-oai
Copy link
Collaborator Author

@codex review this

Read active handoff id when sending realtime handoff output and keep queue payload to text-only.
Copy link
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6476bd42f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +2398 to +2399
if !matches!(msg, EventMsg::TurnComplete(_)) {
return;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Clear active handoff on aborted or failed turns

maybe_clear_realtime_handoff_for_event only clears state on EventMsg::TurnComplete. If a delegated turn is interrupted/errors (TurnAborted, stream failure, etc.), active_handoff_id remains set. Later assistant output still passes maybe_mirror_event_text_to_realtime and gets sent to conversation.handoff.append, misrouting unrelated text to a stale handoff.

Useful? React with 👍 / 👎.

Comment on lines 259 to +260
pub async fn send_audio_frame(&self, frame: RealtimeAudioFrame) -> Result<(), ApiError> {
self.send_json(RealtimeOutboundMessage::InputAudioDelta {
delta: frame.data,
sample_rate: frame.sample_rate,
num_channels: frame.num_channels,
samples_per_channel: frame.samples_per_channel,
})
.await
self.send_json(RealtimeOutboundMessage::InputAudioBufferAppend { audio: frame.data })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve input audio format when appending realtime audio

send_audio_frame now transmits only frame.data and drops sample_rate/num_channels, while session.update fixes input format to 24k mono. Callers can still submit arbitrary RealtimeAudioFrame formats, so non-24k/non-mono audio will be interpreted with the wrong format, causing distorted or incorrect transcription/audio behavior.

Useful? React with 👍 / 👎.

aibrahim-oai and others added 4 commits March 2, 2026 14:36
Replace the unused handoff_id guard in send_output with an explicit active-state check.
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
@aibrahim-oai aibrahim-oai merged commit b20b6aa into main Mar 3, 2026
28 of 31 checks passed
@aibrahim-oai aibrahim-oai deleted the realtime-api-migration branch March 3, 2026 00:05
@github-actions github-actions bot locked and limited conversation to collaborators Mar 3, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants