Skip to content

Update realtime handoff transcript handling#18597

Merged
guinness-oai merged 6 commits intomainfrom
guinness/realtime-attach-transcript-delta
Apr 20, 2026
Merged

Update realtime handoff transcript handling#18597
guinness-oai merged 6 commits intomainfrom
guinness/realtime-attach-transcript-delta

Conversation

@guinness-oai
Copy link
Copy Markdown
Collaborator

@guinness-oai guinness-oai commented Apr 19, 2026

Summary

This PR aims to improve integration between the realtime model and the codex agent by sharing more context with each other. In particular, we now share full realtime conversation transcript deltas in addition to the delegation message.

realtime_conversation.rs now turns a handoff into:

<realtime_delegation>
  <input>...</input>
  <transcript_delta>...</transcript_delta>
</realtime_delegation>

Implementation notes

The transcript is accumulated in the realtime websocket layer as parsed realtime events arrive. When a background-agent handoff is requested, the current transcript snapshot is copied onto the handoff event and then serialized by realtime_conversation.rs into the hidden realtime delegation envelope that Codex receives as user-turn context.

For Realtime V2, the session now explicitly enables input audio transcription, and the parser handles the relevant input/output transcript completion events so the snapshot includes both user speech and realtime model responses. The delegation <input> remains the actual handoff request, while <transcript_delta> carries the surrounding conversation history for context.

Reviewers should note that the transcript payload is intended for Codex context sharing, not UI rendering. The realtime delegation envelope should stay hidden from the user-facing transcript surface, while still being included in the background-agent turn so Codex can answer with the same conversational context the realtime model had.

@guinness-oai
Copy link
Copy Markdown
Collaborator Author

@codex

@guinness-oai guinness-oai marked this pull request as ready for review April 19, 2026 23:07
Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bda8b952ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +560 to +561
if text.is_empty() || contains_transcript_entry(entries, role, &text) {
return;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve repeated utterances in active transcript

Avoid de-duplicating transcript entries by (role,text) across the whole history. contains_transcript_entry causes later repeated phrases (e.g. user says the same request again) to be dropped, so active_transcript is no longer a full chronological transcript and handoff context can omit the latest utterance.

Useful? React with 👍 / 👎.

Comment on lines 131 to 134
item.get("id")
.and_then(Value::as_str)
.map(str::to_string)
.map(|item_id| RealtimeEvent::ConversationItemDone { item_id })
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore transcript handling for conversation.item.done

conversation.item.done no longer maps message content to Input/OutputTranscriptDone; it now returns only ConversationItemDone. When providers send finalized text only in this event, transcript text is lost because downstream transcript accumulation ignores ConversationItemDone, breaking handoff context completeness.

Useful? React with 👍 / 👎.

@guinness-oai guinness-oai merged commit 126bd6e into main Apr 20, 2026
25 checks passed
@guinness-oai guinness-oai deleted the guinness/realtime-attach-transcript-delta branch April 20, 2026 21:04
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants