fix(xai): emit one final user transcript per turn - #6729
Merged
Conversation
xAI sends the transcript of an input item through conversation.item.input_audio_transcription.completed at every audio commit, and the item outlives the pause that caused the commit. Each event re-transcribes the item from the start, wording and punctuation included, so finalizing on a commit delivered the same words again in a slightly different form. Hold the transcript instead and deliver the last one when the response speaks, which is the only signal that the turn ended. Three faults around the same behaviour go with it. A response the user talks over is dropped with no output and no response.done, so its streams stayed open until the interruption timeout cancelled the speech five seconds later. An item anchored to one xAI never announced was rejected by the mirror, which stranded every later item behind it. A rejected item event never gets its conversation.item.deleted reply, so update_chat_ctx waited out its own timeout inside the speech.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
theomonnom
approved these changes
Aug 6, 2026
chenghao-mou
reviewed
Aug 6, 2026
An error names the client event that drew it, while conversation.item added and deleted name only the item, so the waiters need both indexes. The event id map now holds the future itself, and every setter checks that it is not already done, so a rejected delete no longer fails the create still in flight beside it. The waiters are dropped in a finally, so neither a rejection nor a cancellation leaves one parked under its item id. A fatal error keeps its old path. The rejection settles its future and then falls through to the raise, so an exhausted quota still stops the session instead of being swallowed as a rejected item. Reconnect resets the input turn state before it snapshots the mirror, since the xAI override flushes the held transcript there. The turn was otherwise replayed to the new session without its text.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
The event id map outlived the update that filled it, so an error arriving after the update gave up popped a waiter that was already cancelled, settled nothing, and returned before the log and the emit. A slow server rejection went unreported, where it used to surface as a recoverable error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6710
xAI sends the transcript of an input item through
conversation.item.input_audio_transcription.completedat every audio commit, and the item stays open across the pause that caused the commit.Each of those events transcribes the item again from the start, punctuation and wording with it, so a final on every commit sent the same words back in a new form. One turn became several: repeated
user_input_transcribedevents, a duplicate message insession.history, and a new transcript segment on the client each time.The plugin now holds the transcript and delivers the last one when the response speaks. That is the only signal xAI gives that the turn ended: it opens a response at every commit and drops the ones the user talks over, with no output and no
response.done.What else this corrects
A dropped response left its streams open, so the speech waiting on them ended only when the interruption timeout cancelled it after five seconds. It is now closed when the interruption sends its cancel.
An item anchored to one that xAI announced and then dropped was rejected by the remote chat context, which stranded every later item behind it. It is appended instead, with a warning.
A rejected item event never gets the
conversation.item.deletedreply that settles it, soupdate_chat_ctxwaited out its own five second timeout. Because the interrupted path awaits that update inside the speech, it took the speech down with it. The future now fails on the rejection, andupdate_chat_ctxlogs a warning and continues. This part is in the openai plugin, since nothing about it is specific to xAI.Also adds
grok-voice-think-fast-2.0to the supported realtime models.Alternative
#6711 proposed fixing this in
RoomIO, by keying the transcript segment onitem_id. It is closed in favor of this one, because it reaches only the room transcription output: the console and any otherTextOutputstill see the repeated finals, as doAgentSession.historyand the remote chat context.