Skip to content

fix(research): preserve Campaign checkpoints across interruptions - #1100

Merged
proerror77 merged 3 commits into
mainfrom
codex/campaign-recovery-20260905
Sep 5, 2026
Merged

fix(research): preserve Campaign checkpoints across interruptions#1100
proerror77 merged 3 commits into
mainfrom
codex/campaign-recovery-20260905

Conversation

@proerror77

@proerror77 proerror77 commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Change

Recover a Campaign controller interrupted after learning publication or before completion cleanup without changing immutable evidence or entering an unapproved child. Previously, rerunning the learner could change reused_existing in an already-published report, while deleting the request before the completion marker could make the finalized generation impossible to resume.

Persist and validate the first complete learn report and plan before publication. Commit a versioned generation completion record before sensitive-file cleanup, rebuild the cycle summary from that record, and bind completed ACK retries to their original Pod. Status and resume reject corrupted or unsupported completion records. The Monorepo CI control-plane check now invokes the controller syntax, lint, and recovery contract checks so these regressions are exercised remotely.

Issue relationship

None

Validation

At b28d39aaccc65d6de213fbd7849c221136db181b, including main 8c7d05cb617c9516fbde571c5676285e887f0ed5:

  • Reproduced the immutable report mismatch and missing-request recovery failures before the implementation change.
  • deployment/aliyun/research/test-campaign-cycle-controller.sh: existing contract flow and 13 added recovery/conflict scenarios passed, including process termination at cleanup/completion boundaries, lost PUT responses, summary-write failure, late parent retry, bounded/no-improvement stops, and local/remote evidence corruption.
  • bash -n deployment/aliyun/research/scripts/campaign-cycle-controller.sh deployment/aliyun/research/test-campaign-cycle-controller.sh: passed.
  • shellcheck -e SC2154 deployment/aliyun/research/scripts/campaign-cycle-controller.sh deployment/aliyun/research/test-campaign-cycle-controller.sh: passed, retaining the existing ERR-trap exclusion.
  • git diff --check: passed.
  • Actionlint for .github/workflows/ci.yml passed; the controller-script path selects control=true, activating the new CI step. Current-head remote CI is pending.

Runtime impact and rollback

Research control-plane checkpoint behavior only. No cloud resource, model, fee, Gate, holdout, order, risk, or paused runtime was changed.

The completion checkpoint format intentionally rejects old empty markers. Preserve historical evidence and use a fresh cycle for acceptance; resuming an older checkpoint requires a separately reviewed migration. Roll back source before adopting the new format, or retain the matching controller version for in-progress new-format cycles; an old controller must not resume them.

Local fault injection uses a local filesystem and fake external ports. It does not prove node-power-loss durability, OSS CSI rename atomicity, or multi-writer fencing. New-image publication, the frozen 31-hour inventory at 12 GiB, and a real parent/child ACK readback remain separate pending acceptance.

@coderabbitai

coderabbitai Bot commented Sep 5, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 1a36ebf6-9ab7-4213-8517-3197223ee743


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 5, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-05T03:34:55.848424Z da61e0a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@proerror77
proerror77 merged commit 26c3da1 into main Sep 5, 2026
43 checks passed
@proerror77
proerror77 deleted the codex/campaign-recovery-20260905 branch September 5, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant