Skip to content

Conversation

thli-openai
Copy link
Contributor

  1. Replaced deprecated ScriptProcessor with audio worklets for both recording and playback
  2. Sanitized history_updated events to remove audio bytes from bloating the websocket events to the client
  3. Swapped default realtime config to use server vad for snappier interruptions

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting

Comment on lines +574 to +586
async ensurePlaybackNode() {
if (this.playbackNode) {
return;
}

this.isPlayingAudio = true;

// Initialize audio context if needed
if (!this.playbackAudioContext) {
this.playbackAudioContext = new AudioContext({ sampleRate: 24000, latencyHint: 'interactive' });
}
if (!this.playbackInitPromise) {
this.playbackInitPromise = (async () => {
if (!this.playbackAudioContext) {
this.playbackAudioContext = new AudioContext({ sampleRate: 24000, latencyHint: 'interactive' });
}

// Ensure context is running (autoplay policies can suspend it)
if (this.playbackAudioContext.state === 'suspended') {
try { await this.playbackAudioContext.resume(); } catch {}
}
if (this.playbackAudioContext.state === 'suspended') {
try { await this.playbackAudioContext.resume(); } catch {}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Resume playback AudioContext after browser suspends it

When playAudio pushes a chunk it calls ensurePlaybackNode, but this helper immediately returns once this.playbackNode has been created. That means the call to AudioContext.resume() only happens the first time the node is built. Browsers (Chrome/Edge/Safari) automatically suspend idle audio contexts after a period of silence, so any later audio events will enqueue chunks into the worklet while the context remains suspended and nothing is heard. The previous implementation resumed the context before every playback, so this is a regression that makes audio permanently silent after the tab has idled. Consider resuming the context even when playbackNode already exists or resuming inside playAudio before posting chunks.

Useful? React with 👍 / 👎.

@seratch seratch added documentation Improvements or additions to documentation feature:realtime labels Oct 16, 2025
@seratch
Copy link
Member

seratch commented Oct 16, 2025

Can you fix this mypy error?

Run make mypy
uv run mypy . --exclude site
examples/realtime/app/server.py:136: error: Returning Any from function declared to return "dict[str, Any]"  [no-any-return]
Found 1 error in 1 file (checked 342 source files)
make: *** [Makefile:20: mypy] Error 1
Error: Process completed with exit code 2.

@thli-openai thli-openai requested a review from seratch October 16, 2025 16:28
Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a few manual tests and didn't see any issues. This improvement is fantastic 👍

@seratch seratch merged commit d9f1d5f into main Oct 17, 2025
5 checks passed
@seratch seratch deleted the fix/realtime-example branch October 17, 2025 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feature:realtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants