fix(deep-research): recover heavy DR report from connector widget state - #9
Merged
Merged
Conversation
ChatGPT moved heavy Deep Research to the "Deep Research App" connector
(connectors://connector_openai_deep_research), which renders the report in an
embedded widget and never writes it as an assistant text node. The old
_poll_dr_completion scanned only assistant text, so heavy runs timed out at
1800s with an empty report even though the research completed server-side.
The report lives in the hidden widget state (widget_state.report_message).
_poll_dr_completion now fetches the conversation with
?include_visually_hidden_messages=true&include_widget_state=true and recovers
the report (text + content_references) from either widget-state carrier — a
"The latest state of the widget is: {…}" tool node, or
message.metadata.chatgpt_sdk.widget_state — via the new
_dr_report_from_widget_state helper. Existing citation-extraction is untouched.
Verified by recovering three real completed reports headlessly
(45.6K / 52.4K / 51.5K chars, with citations). Adds 4 fixture-based tests
(real oracle, no network). Light deep_research uses a different (SearchGPT)
mechanism and is unchanged; a dedicated light-mode fix is tracked as a follow-up.
Bumps version 0.0.3 -> 0.0.4; updates deep-research skill doc + CHANGELOG.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThis PR fixes heavy Deep Research report extraction by recovering the final report from the connector widget state instead of timing out while polling assistant text nodes. It adds widget-state parsing helpers, integrates them into the polling loop, validates with tests, and documents the corrected behavior with version bumps. ChangesHeavy Deep Research widget-state report recovery
Possibly related PRs
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
gpt2agent/skills/deep-research/SKILL.md (1)
159-164:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate stale heavy-DR recovery instructions in this doc.
Lines 159-164 still direct users to recover via assistant text nodes and state heavy may be unrecoverable, which conflicts with the new widget-state recovery behavior documented above in this same file.
Suggested doc correction
--mapping[*].message` for the newest assistant text node with status --`finished_successfully` — its `metadata.content_references` holds the citation --URLs. NOTE: heavy DR via the connector may render an "embedded UI experience" --and never write a fetchable report node back; in that case there is nothing to --recover and the run must be redone in a quiet window. Wait for the rate limit to --ease first — repeated GETs while rate-limited keep it hot. +`mapping[*].message` for widget-state carriers and recover from +`widget_state.report_message.content.parts[0]` plus +`report_message.metadata.content_references` (request with +`?include_visually_hidden_messages=true&include_widget_state=true`). +Use assistant-text fallback only if widget state is absent.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@gpt2agent/skills/deep-research/SKILL.md` around lines 159 - 164, Update the stale recovery guidance that tells users to read assistant text nodes via BackendClient/get_conversation and inspect mapping[*].message/metadata.content_references for citations; instead, align this section with the new widget-state recovery behavior by removing the assertion that heavy-DR runs are unrecoverable and add instructions to use the widget-state recovery flow described earlier (referencing "widget-state recovery" and the heavy-DR connector behavior) so readers are directed to the correct recovery mechanism rather than extracting content from assistant text nodes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@gpt2agent/sse.py`:
- Around line 1552-1554: The code currently emits the full widget_text in a
"progress" chunk which causes duplicate content for append-based consumers;
change it to send only the delta since last_emitted (e.g., compute delta =
widget_text[len(last_emitted):] or similar) and yield {"type":"progress","text":
delta} only when delta is non-empty, then update last_emitted to widget_text;
apply this change in the same block that currently yields the full widget_text
so the "progress" chunks match the existing delta semantics used elsewhere in
the method.
---
Outside diff comments:
In `@gpt2agent/skills/deep-research/SKILL.md`:
- Around line 159-164: Update the stale recovery guidance that tells users to
read assistant text nodes via BackendClient/get_conversation and inspect
mapping[*].message/metadata.content_references for citations; instead, align
this section with the new widget-state recovery behavior by removing the
assertion that heavy-DR runs are unrecoverable and add instructions to use the
widget-state recovery flow described earlier (referencing "widget-state
recovery" and the heavy-DR connector behavior) so readers are directed to the
correct recovery mechanism rather than extracting content from assistant text
nodes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 02e6851e-4a8a-4d62-ab11-a9f2f29aa3a5
📒 Files selected for processing (7)
CHANGELOG.mdgpt2agent/skills/deep-research/SKILL.mdgpt2agent/skills/deep-research/bin/deep_research.pygpt2agent/sse.pypyproject.tomltests/fixtures/heavy_dr_widget_state.jsontests/test_heavy_dr_parser.py
Comment on lines
+1552
to
+1554
| if widget_text != last_emitted: | ||
| yield {"type": "progress", "text": widget_text} | ||
| yield { |
There was a problem hiding this comment.
Keep widget-state progress chunks delta-based.
Line 1553 emits full widget_text as progress. If seed_text is already present, append-based consumers will duplicate content. Match the existing delta semantics used elsewhere in this method.
Proposed patch
if widget_text:
if widget_text != last_emitted:
- yield {"type": "progress", "text": widget_text}
+ if widget_text.startswith(last_emitted):
+ delta = widget_text[len(last_emitted) :]
+ if delta:
+ yield {"type": "progress", "text": delta}
+ else:
+ yield {"type": "progress", "text": widget_text}
+ last_emitted = widget_text
yield {
"type": "done",
"text": widget_text,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if widget_text != last_emitted: | |
| yield {"type": "progress", "text": widget_text} | |
| yield { | |
| if widget_text: | |
| if widget_text != last_emitted: | |
| if widget_text.startswith(last_emitted): | |
| delta = widget_text[len(last_emitted) :] | |
| if delta: | |
| yield {"type": "progress", "text": delta} | |
| else: | |
| yield {"type": "progress", "text": widget_text} | |
| last_emitted = widget_text | |
| yield { | |
| "type": "done", | |
| "text": widget_text, |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@gpt2agent/sse.py` around lines 1552 - 1554, The code currently emits the full
widget_text in a "progress" chunk which causes duplicate content for
append-based consumers; change it to send only the delta since last_emitted
(e.g., compute delta = widget_text[len(last_emitted):] or similar) and yield
{"type":"progress","text": delta} only when delta is non-empty, then update
last_emitted to widget_text; apply this change in the same block that currently
yields the full widget_text so the "progress" chunks match the existing delta
semantics used elsewhere in the method.
This was referenced Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause — heavy DR moved to a connector widget
ChatGPT migrated heavy Deep Research to the "Deep Research App" connector (pineapple URI
connectors://connector_openai_deep_research). The connector runs the research server-side and renders the report in an embedded widget; it never writes the report as an assistant text node in the conversationmapping. The assistant text node for the DR turn stays 0-char.The old
_poll_dr_completionwalkedmapping[*].messageonly for assistant text nodes, found nothing non-empty, and timed out at 1800s — even thoughvenus_widget_state.statuswas alreadycompleted.The fix — recover from the hidden widget state
The full report actually lives in the conversation, but only when you ask for the hidden/widget data:
This exposes the report at
widget_state.report_message.content.parts[0](the full Markdown) plusreport_message.metadata.content_references(source URLs, already in theitems[].url/titleshape the runner renders). The widget state is delivered via two carriers, both handled by the new_dr_report_from_widget_statehelper:"The latest state of the widget is: {…}", andmessage.metadata.chatgpt_sdk.widget_state(a JSON string)._poll_dr_completionnow (a) requests the conversation with those two params and (b) checks the widget state each poll, emitting the report as adoneevent the moment it appears. Existing assistant-text + citation-extraction logic is untouched (fallback path preserved).Test evidence (real, by execution)
Recovered 3 real completed reports headlessly, zero DR quota spent, through the patched code path:
(dr3 has no grouped-source list — that report wrote 128 source URLs inline in the prose; the text is complete.)
Full suite (excluding the pre-existing live-network flake
test_account_status_has_subscription, which 401s identically on cleanmain):Adds 4 fixture-based tests (
tests/fixtures/heavy_dr_widget_state.json, real oracle, no network), incl.test_poll_completion_recovers_widget_reportdriving the full async poll path:Scope
deep_research_heavy: fixed (this PR).deep_research(model=research, SearchGPT backend): a different mechanism, not changed here. Its longest-donemitigation from 0.0.3 remains; a dedicated light-mode fix is tracked as a follow-up (see CHANGELOG Notes + SKILL.md TODO).Bumps
0.0.3 → 0.0.4; updates the deep-research skill doc + CHANGELOG.🤖 Generated with Claude Code
Summary by CodeRabbit