Skip to content

fix(responses): repair sparse terminal output for Grok Build - #3756

Merged
lidge-jun merged 16 commits into
devfrom
codex/release-244-grok-terminal-07c0
Sep 6, 2026
Merged

fix(responses): repair sparse terminal output for Grok Build#3756
lidge-jun merged 16 commits into
devfrom
codex/release-244-grok-terminal-07c0

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

Co-authored-by: Maple hzlhu@qq.com

Verification

  • Current head eaa005ae0ba4ec48e8106df70b7c2f0b286ef32b includes the stored Pool recovery boundary repair and is awaiting its own hosted CI. Earlier green runs do not verify this head. No local tests, typecheck or build were run.

  • Independent plan audit passed. Source/protocol/module review is in progress; hosted exact-head CI will run asynchronously.

  • Unit and server fixtures cover text/function/custom-plus-message recovery, invalid call IDs, bounds, terminal precedence, provider-repair coexistence and the same-provider marker/absent-marker control.

  • Existing provider snapshot test bodies and the f121348 sparse JSON/function-repair replay test are unchanged.

  • No local test suite, typecheck or build was run. A local static AST scan could not start because TypeScript is not installed; no dependency was installed and no static scan success is claimed. No live Kiro or Grok request was used.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Native stack #3759 was dissolved at the owner's explicit request. The PR head is preserved; integration continues bottom-up with ordinary PRs and admin merge after current-head CI.

Summary by CodeRabbit

  • New Features
    • Encrypted combo tasks can now recover once and route to an available fallback target when no usable native target remains.
    • Grok clients can reconstruct missing or sparse terminal Responses output when valid completed items are available.
  • Bug Fixes
    • Improved cancellation and failure handling prevents unreadable content from being routed or plaintext from being persisted.
    • Canonical native targets are no longer incorrectly excluded solely due to cached quota exhaustion.
  • Documentation
    • Updated configuration and adapter documentation across supported languages to describe encrypted combo recovery and Grok snapshot repair.

t and others added 13 commits September 6, 2026 16:30
…tion

Carry #3706 onto the verified opaque recovery foundation, keep canonical quota ownership and preserve caller cancellation at both recovery sites.

Co-authored-by: yxr1995-maker <257504378+yxr1995-maker@users.noreply.github.com>
…ated items

Carry #3388 with a dedicated tracker and shared leaf codec; preserve existing provider repair, stream ordering and unmarked-client behavior.

Co-authored-by: Maple <hzlhu@qq.com>
@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: 76740c30-631d-4c06-bc21-6f184598021a

📥 Commits

Reviewing files that changed from the base of the PR and between 96094c3 and eaa005a.

📒 Files selected for processing (25)
  • devlog/_plan/260906_release_244_followups/000_plan.md
  • devlog/_plan/260906_release_244_followups/050_combo_recovery.md
  • devlog/_plan/260906_release_244_followups/051_combo_recovery_implementation.md
  • devlog/_plan/260906_release_244_followups/060_grok_terminal.md
  • docs-site/src/content/docs/fr/reference/configuration/agents.md
  • docs-site/src/content/docs/guides/sub-agent-surface.md
  • docs-site/src/content/docs/ja/reference/configuration/agents.md
  • docs-site/src/content/docs/ko/reference/configuration/agents.md
  • docs-site/src/content/docs/reference/adapters.md
  • docs-site/src/content/docs/reference/configuration/agents.md
  • docs-site/src/content/docs/ru/reference/configuration/agents.md
  • docs-site/src/content/docs/tr/reference/configuration/agents.md
  • docs-site/src/content/docs/zh-cn/reference/configuration/agents.md
  • docs-site/src/content/docs/zh-tw/reference/configuration/agents.md
  • src/combos/resolve.ts
  • src/server/grok-responses-snapshot-repair.ts
  • src/server/responses-snapshot-codec.ts
  • src/server/responses-snapshot-repair.ts
  • src/server/responses/core.ts
  • structure/04_transports-and-sidecars.md
  • tests/codex-integration/combos.test.ts
  • tests/responses/responses-pool-401-refresh.test.ts
  • tests/responses/responses-snapshot-repair-server.test.ts
  • tests/responses/responses-snapshot-repair.test.ts
  • tests/server/agent-task-recovery-combo.test.ts

📝 Walkthrough

Walkthrough

The PR adds deferred encrypted combo recovery, canonical provider quota handling, and cancellation coverage. It also adds Grok-specific sparse terminal snapshot repair, shared snapshot helpers, extensive validation tests, localized documentation, and release follow-up plans.

Changes

Recovery and snapshot repair

Layer / File(s) Summary
Deferred combo recovery and provider selection
src/combos/resolve.ts, src/server/responses/core.ts, tests/codex-integration/combos.test.ts, tests/server/agent-task-recovery-combo.test.ts, tests/responses/responses-pool-401-refresh.test.ts
Combo dispatch now performs one-shot encrypted-task recovery only when native target selection cannot proceed. Canonical OpenAI forward providers bypass cached quota exhaustion checks. Tests cover cooldowns, authorization failures, cancellation, replay handling, fail-closed responses, and plaintext cleanup.
Combo behavior documentation and follow-up plans
devlog/_plan/260906_release_244_followups/*, docs-site/src/content/docs/*/reference/configuration/agents.md, docs-site/src/content/docs/guides/sub-agent-surface.md
Plans and localized guides describe mixed combo recovery, cancellation behavior, quota filtering, fail-closed routing, and asynchronous CI scheduling.
Grok sparse snapshot repair implementation
src/server/grok-responses-snapshot-repair.ts, src/server/responses-snapshot-codec.ts, src/server/responses-snapshot-repair.ts, src/server/responses/core.ts
The server adds bounded validation and reconstruction of sparse Grok terminal snapshots. Shared wire helpers move into a codec module. Core enables the rewrite for Grok requests.
Grok repair validation and architecture documentation
tests/responses/responses-snapshot-repair.test.ts, tests/responses/responses-snapshot-repair-server.test.ts, docs-site/src/content/docs/reference/adapters.md, structure/04_transports-and-sidecars.md
Tests cover valid output items, malformed or contradictory evidence, contiguous indices, call identifiers, budget release, disposal, composition, and client-marker behavior. Documentation records module ownership and fail-closed rules.

Estimated code review effort: 4 (Complex) | ~60 minutes

Suggested reviewers: invalid-email-address

Sequence Diagram(s)

Encrypted combo recovery

sequenceDiagram
  participant Caller
  participant handleComboResponses
  participant ComboResolver
  participant RecoveryHelper
  participant RoutedProvider
  Caller->>handleComboResponses: submit encrypted NEW_TASK
  handleComboResponses->>ComboResolver: select eligible native target
  ComboResolver-->>handleComboResponses: no eligible native target
  handleComboResponses->>RecoveryHelper: recover task once
  RecoveryHelper-->>handleComboResponses: readable task
  handleComboResponses->>RoutedProvider: dispatch recovered task
Loading

Grok terminal snapshot repair

sequenceDiagram
  participant GrokClient
  participant ResponsesCore
  participant GrokSnapshotRewrite
  participant ResponsesStream
  GrokClient->>ResponsesCore: send x-opencodex-grok: 1 request
  ResponsesCore->>GrokSnapshotRewrite: enable terminal repair
  ResponsesStream->>GrokSnapshotRewrite: send output_item.done events
  GrokSnapshotRewrite-->>GrokClient: send reconstructed response.completed output
Loading
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/release-244-grok-terminal-07c0

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Sep 6, 2026
@lidge-jun

Copy link
Copy Markdown
Owner Author

리뷰 · 우선순위 72 / 80

이 PR은 release-244 플랜의 060_grok_terminal 칸입니다. 베이스가 지금 dev(HEAD cededd5ad, #3755까지)가 아니라 아직 열린 draft #3754 브랜치 codex/release-244-combo-recovery-07c0입니다. Grok Build는 스트림 델타를 그려도, 마지막 response.completed.response.output이 비어 있거나 아예 없으면 Assistant 최종 항목을 못 만듭니다. 이 PR은 x-opencodex-grok: 1로 surface가 grok인 요청에만, 진짜 output_item.done 증거로 그 빈 터미널을 다시 채웁니다. 원본 캐리는 #3388(지금도 open draft)이고, Co-authored-by에 Maple(zleo-ai)이 있습니다. 이 PR 자체도 draft입니다.

지금 HEAD의 rewrite 순서는 custom-tool restore → tool-search restore → Copilot → (여기 없음) provider snapshot → field backfill → function repair → undeclared-tool guard입니다. 이 PR은 그 사이에 Grok tracker를 provider snapshot 바로 앞에 끼웁니다. 새 파일 src/server/grok-responses-snapshot-repair.ts가 클라이언트 전용 정책을 갖고, 기존 src/server/responses-snapshot-repair.ts에서 isPlainObject / jsonBlock / RetainedOutputItem만 잎 모듈 src/server/responses-snapshot-codec.ts로 빼서 두 tracker가 공유합니다. 공개 snapshot export는 그대로이고, 서로 circular import도 없습니다. 마커는 src/server/index.ts / chat-completions.ts가 헤더를 보면 logCtx.surface = "grok"로 바꾸는 기존 경로이고, 인증·권한이 아닙니다. docs-site adapters와 structure/04_transports-and-sidecars.md에도 그 경계를 적어 두었습니다.

복구 규칙은 빡셉니다. done 항목은 타입별 좁은 validator를 통과해야 하고, index는 유일·연속·0부터여야 하며, open 항목이 남아 있거나 tainted면 손대지 않습니다. hasVisibleOutput이 있어야만 채웁니다. message의 nonempty output_textfunction_call만 visible이고, custom_tool_call / reasoning / web_search / code_interpreter / mcp는 보관만 하고 visible로는 안 칩니다. 그래서 custom-tool만 있는 스트림은 빈 터미널을 채우지 않습니다(의도된 fail-closed). nonempty 터미널은 권위 있고, failed/incomplete는 completed로 바꾸지 않습니다. call_id가 비어 있거나 공백이면 function/custom을 거절하는 테스트도 같이 들어 있습니다. 기존 f121348 sparse JSON/function-repair 서버 픽스처는 유지합니다.

플랜 000_plan.md에 적힌 대로 구현·리뷰는 호스트 CI를 기다리지 않고 다음 칸으로 가지만, 머지·릴리스는 final-head 초록이 필요합니다. 지금 이 head의 linux test 1–4 / macos / gates / keyring / npm-global은 초록으로 보이지만, Windows 6-shard는 skipping으로 보이고 본문 security 체크는 비어 있습니다. 로컬 스위트·라이브 Grok/Kiro는 없습니다. types/config 대분할과 무관합니다. #3388은 이 캐리가 랜딩하면 close-don't-rebase leftover입니다. #3506(Cursor/Grok semantic no-progress)과는 범위가 다릅니다.

우선순위 72인 이유다. combo 다음 필수 칸이고 Grok Build 빈 completed 출력 구멍을 막습니다. 다만 #3754 draft 스택 의존, draft 게이트, Windows final-head·security 체크 미완이 남아서 #3754(73)보다 한 칸 낮춥니다. 부모 없이 올리면 깨집니다.

경로/심볼 - src/server/responses/core.ts grokClientSnapshotRepairEnabled / rewrite 삽입 위치(provider snapshot 직전)
경로/심볼 - src/server/grok-responses-snapshot-repair.ts createGrokResponsesSparseTerminalBlockRewrite / trustedGrokCompletedItem / hasVisibleOutput
경로/심볼 - src/server/responses-snapshot-codec.ts 공유 leaf · responses-snapshot-repair.ts import 전환
경로 - tests/responses/responses-snapshot-repair.test.ts / responses-snapshot-repair-server.test.ts
경로 - docs-site reference/adapters.md · structure/04_transports-and-sidecars.md · plan 060_grok_terminal.md
경로/심볼 - 부모 #3754 / leftover #3388 / 마커 x-opencodex-groklogCtx.surface === "grok" / security checklist 미체크 · draft

메인테이너의 판단이 필요한 지점

  • #3754 Ready·머지 직후에만 이 PR을 Ready/머지할지(권장 순서)
  • Windows 6-shard와 security 체크를 머지 전 final-head 기준으로 채울지(비동기 CI 정책과 충돌 없는지)
  • custom-only 터미널을 계속 fail-closed로 둘지(지금 visible 게이트)
  • 머지 후 #3388Landed via #3756로 닫을지

너의 추천
#3754를 먼저 Ready·머지한 뒤, 이 PR을 Ready로 바꾸고 exact-head(특히 Windows) 초록이면 머지하세요. #3388은 close-don't-rebase leftover로 랜딩 코멘트 후 닫으세요. 라이브 Grok/Kiro 금지. types/config 이슈 아님. 다음 칸은 quota-proxy docs(#3758 / 070).

이 댓글은 grok-bot이 작성했습니다

Base automatically changed from codex/release-244-combo-recovery-07c0 to dev September 6, 2026 10:07
@lidge-jun
lidge-jun marked this pull request as ready for review September 6, 2026 10:09
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner September 6, 2026 10:09
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-06T10:14:55.329255Z eaa005a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@lidge-jun
lidge-jun enabled auto-merge (squash) September 6, 2026 10:09
@lidge-jun
lidge-jun merged commit a4451e8 into dev Sep 6, 2026
37 of 40 checks passed
@lidge-jun
lidge-jun deleted the codex/release-244-grok-terminal-07c0 branch September 6, 2026 10:11

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eaa005ae0b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +23 to +27
"custom_tool_call",
"web_search_call",
"code_interpreter_call",
"mcp_call",
]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle restored tool-search calls in terminal repair

When a marked Grok request declares tool_search on a noncanonical Responses route, createRoutedToolSearchRestoreBlockRewrite runs immediately before this tracker and rewrites the added/done item to tool_search_call. That type is absent from this allowlist, so the added event taints the tracker and the done event is also rejected; a sparse terminal therefore remains empty and Grok can repeat the billable turn. Validate and retain the restored type, counting it as visible client work, while preserving the final client-facing shape.

AGENTS.md reference: src/AGENTS.md:L19-L19

Useful? React with 👍 / 👎.

Comment on lines +2823 to 2826
// Keep the spent Pool budget sticky even after a recovered routed child:
// no later failure may reopen ordinary combo/native account hopping.
adoptFailedChildLog(childLog);
return lastFailure;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cool a failed routed target before the sticky return

After a stored Pool 401 refresh/replay is exhausted, this branch can recover the encrypted task and continue with a routed target. If that target then returns a hop-class failure such as 429 or 503, the sticky flag remains set and this return bypasses advanceComboAfterFailure, which is where noteComboFailure and coolComboTarget run. The failed backup consequently remains immediately selectable on later child requests instead of respecting combo cooldown/failover; record its failure and cooldown while still suppressing additional same-request hops.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant