Skip to content

fix(realtime): clamp interrupt truncation to received audio - #4122

Merged
seratch merged 1 commit into
openai:mainfrom
hsusul:fix/realtime-interrupt-clamp-audio-end-ms
Aug 2, 2026
Merged

fix(realtime): clamp interrupt truncation to received audio#4122
seratch merged 1 commit into
openai:mainfrom
hsusul:fix/realtime-interrupt-clamp-audio-end-ms

Conversation

@hsusul

@hsusul hsusul commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

OpenAIRealtimeWebSocketModel._send_interrupt() can send a conversation.item.truncate whose audio_end_ms is larger than the audio duration the client actually received, which the Realtime API rejects. The value is now clamped to the received audio length, matching what the VAD (input_audio_buffer.speech_started) barge-in path already does.

Affected component: src/agents/realtime/openai_realtime.py — explicit interrupts (RealtimeSession.interrupt() / RealtimeModelSendInterrupt).

Problem. #2370 reported that the SDK sends truncate events with an audio_end_ms bigger than the item's audio length, so the server answers with an error. #2374 fixed both interrupt paths by clamping truncated_ms to the received audio length. The follow-up refactor in #2385 restructured both paths so the truncate decision no longer depends on _ongoing_response alone; the VAD path kept truncated_ms = min(truncated_ms, max_audio_ms), but the _send_interrupt() path lost it while the surrounding else: block was removed. Since then, whenever a response is still in progress, an explicit interrupt sends the raw elapsed time.

This is easy to hit with the default timing. Without a custom RealtimePlaybackTracker, elapsed_ms is wall-clock time since the first audio delta for the item, while max_audio_ms is the audio actually received. Any pause between deltas (model still generating, tool call, slow network) makes elapsed time outgrow the received audio, so the truncate is sent out of range.

Minimal reproduction (no API key, no live request):

model = OpenAIRealtimeWebSocketModel()
model._ongoing_response = True
model._send_raw_message = AsyncMock()
model._audio_state_tracker.set_audio_format("pcm16")

# 48_000 bytes of PCM16 at 24kHz == ~1000 ms of received audio.
with patch("agents.realtime._default_tracker.time.monotonic", return_value=100.0):
    model._audio_state_tracker.on_audio_delta("item_1", 0, b"a" * 48_000)

with patch("agents.realtime.openai_realtime.time.monotonic", return_value=105.0):
    await model._send_interrupt(RealtimeModelSendInterrupt())

Current behavior: the truncate carries audio_end_ms=5000 for an item with 1000 ms of audio. The identical setup driven through input_audio_buffer.speech_started sends audio_end_ms=1000 (this is already asserted by tests/realtime/test_openai_realtime.py::test_speech_started_truncates_when_response_ongoing).

Corrected behavior: both interrupt paths truncate at 1000 ms.

Root cause: the clamp removed from _send_interrupt() in #2385 was never re-added, so only the VAD path bounds truncated_ms by max_audio_ms.

Implementation: restore truncated_ms = min(truncated_ms, max_audio_ms) inside the existing send branch of _send_interrupt(), guarded by max_audio_ms is not None exactly as the VAD path does.

Why this is minimal: four lines in one function. The decision of whether to send a truncate is unchanged (the self._ongoing_response or max_audio_ms is None or truncated_ms < max_audio_ms condition is untouched), only the value is bounded. No public API, no new abstraction, no behavior change when the elapsed time is already within the received audio (the common case), and no change to the VAD path, the playback trackers, or response cancellation.

Test plan

tests/realtime/test_playback_tracker.py:

  • test_interrupt_clamps_truncate_to_received_audio_while_response_ongoing (new): default timing, wall clock 5 s past 1000 ms of received audio, expects audio_end_ms == 1000. Fails on main with 5000 == 1000.
  • test_interrupt_matches_speech_started_truncation_point (new): drives the same tracker state through _send_interrupt() and through input_audio_buffer.speech_started and asserts both truncate at the same point. Fails on main with assert 5000 == 1000.
  • test_interrupt_sends_truncate_when_ongoing_response (existing): still asserts that a truncate is sent while a response is ongoing; the expected value moves from 2000 to 1000 because the custom playback tracker reports 2000 ms for an item with only 1000 ms of received audio. That 2000 expectation was introduced in fix: #2370 send truncate events independent of response state #2385 alongside the dropped clamp.

Execution modes covered: explicit interrupt with the default ModelAudioTracker timing, explicit interrupt with a custom RealtimePlaybackTracker, and VAD speech_started barge-in. Cases where the elapsed time is below the received audio, where no audio state exists (max_audio_ms is None), and where playback is complete and the truncate is skipped are already covered by the existing tests in that file and in tests/realtime/test_openai_realtime.py, and all still pass unchanged.

No lifecycle, cancellation, or cleanup behavior changes: the tracker resets, response.cancel handling, and emitted audio_interrupted events are untouched.

Commands run from the repository root on fc084ae + this change (Python 3.12.13, macOS 15.6 / darwin 24.6.0):

uv run pytest tests/realtime/test_playback_tracker.py -q      -> 11 passed
uv run pytest tests/realtime -q                               -> 360 passed
make format                                                   -> 844 files left unchanged, all checks passed
make lint                                                     -> All checks passed!
make typecheck                                                -> mypy: Success (835 files); pyright: 0 errors, 0 warnings
make tests                                                    -> 6086 passed, 3 skipped (parallel) / 45 passed, 4 skipped (serial)
bash .agents/skills/code-change-verification/scripts/run.sh    -> all commands passed

The three focused tests were also run repeatedly to confirm determinism. Integration test profiles (make integration-tests*) were not run: they require live provider credentials and are unrelated to this change.

Non-goals: no change to the truncate/skip decision, to RealtimePlaybackTracker, to ModelAudioTracker, to response cancellation, or to the VAD path.

Issue number

Follow-up to #2370 (the clamp added in #2374 was dropped from _send_interrupt() by the refactor in #2385).

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

@seratch seratch added this to the 0.19.x milestone Aug 2, 2026
@seratch
seratch enabled auto-merge (squash) August 2, 2026 22:29
@seratch
seratch merged commit bfcfcfc into openai:main Aug 2, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants