fix(xai): emit interim realtime transcripts#1912
fix(xai): emit interim realtime transcripts#1912rosetta-livekit-bot[bot] wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: 59d9e12 The changes in this PR will be included in the next version bump. This PR includes changesets to release 35 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if (event.status === 'in_progress') { | ||
| this.emit('input_audio_transcription_completed', { | ||
| itemId: event.item_id, | ||
| transcript: event.transcript, | ||
| isFinal: false, | ||
| }); | ||
| return; |
There was a problem hiding this comment.
🚩 Failure path produces no final event when only in_progress completed events were received
If xAI sends only completed(in_progress) events (no deltas) and then a transcription.failed event, finalizePartialOnTranscriptionFailure at plugins/openai/src/realtime/realtime_model.ts:1616 will find no accumulator entry (since in_progress doesn't populate it) and return without emitting an isFinal: true event. Consumers waiting for a final transcript would never receive one. In the normal OpenAI flow this doesn't happen because deltas populate the accumulator. This is a narrow edge case (failure after in-progress events from xAI) and may not be worth handling, but reviewers should be aware.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
status: "in_progress"transcription events as interim transcriptsTesting
pnpm build:pluginspnpm test -- plugins/openai/src/realtime/realtime_model.test.tspnpm prettier --check plugins/openai/src/realtime/api_proto.ts plugins/openai/src/realtime/realtime_model.ts plugins/openai/src/realtime/realtime_model.test.tsPort of livekit/agents#6272.
Ported from livekit/agents#6272
Original PR description
Fixes #6271
xAI streams partial transcripts through the same conversation.item.input_audio_transcription.completed event as the final one, distinguishing them with a status field ("in_progress" vs "completed"). The handler delegated to the OpenAI base handler, which ignores status and always emits is_final=True, so every partial was surfaced as a final transcription and produced a duplicate conversation item for a single utterance.
Emit InputTranscriptionCompleted(is_final=False) for in-progress transcripts and only delegate to the base handler (is_final=True) for the final one. A missing status defaults to final.