Skip to content

fix: preserve orchestration sessions across restarts - #3123

Merged
numnx merged 3 commits into
devfrom
fix/sprint-recovery-after-restart
Jul 14, 2026
Merged

fix: preserve orchestration sessions across restarts#3123
numnx merged 3 commits into
devfrom
fix/sprint-recovery-after-restart

Conversation

@numnx

@numnx numnx commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • make QA review verdict-to-fix handoffs durable and resume interrupted task/sprint reviews with the same logical and native provider sessions
  • persist CI-fix and merge-conflict repair checkpoints so restart recovery keeps the same session, workspace, model, baseline, and publication state
  • reconcile successful provider results honestly when shutdown races with terminal bookkeeping, and recover post-provider repair checkpoints as completed
  • harden startup reconciliation, effective workspace-tree recovery, and the orchestration pentest conflict fixture
  • batch Docker inventory lookup during multi-reviewer QA recovery to avoid repeated startup scans
  • document restart guarantees across the canonical and published documentation sets

Root cause

A completed QA changes_requested verdict was durable, but its generated coding handoff was not fully checkpointed before the side effect. After a restart, the scheduler could therefore see neither a fresh completion event nor a recoverable pending handoff and remain at a fixed point. QA, CI-fix, and merge-conflict executions also lacked complete durable continuation metadata across every crash window.

A later CI run exposed a narrower audit race: the repair provider had returned successfully, but shutdown began before its terminal usage row was persisted. The repair was durably published and correctly recovered without duplicate provider work, while startup classified the stale running audit row as cancelled. Durable post-provider checkpoints now prove and restore completed provider status, while genuinely interrupted attempts remain cancelled and continue their preserved session.

Validation

  • pnpm run lint
  • backend: 5,778 passed, 7 skipped
  • dashboard: 2,279 passed, 1 skipped
  • pnpm run build
  • pnpm run audit — no vulnerabilities
  • rapid orchestration suite: 286 passed, 7 skipped
  • full nine-scenario orchestration catalog, including merge-conflict and multi-project override scenarios
  • exact Linux Docker CI DAG command passed three consecutive runs, each with two forced runtime restarts
  • Electron runtime orchestration scenario
  • 129-task DAG pentest: 129/129 completed, zero invariant violations, no stall
  • live restart window: orphaned QA handoff reconciled automatically, all affected tasks completed, and health/readiness remained green

Risk and rollback

Risk is concentrated in startup recovery, provider terminal bookkeeping, and QA/repair state transitions. The new paths are bounded by phase markers, exact invocation correlation, ancestry/tree validation, and retry caps. Roll back by reverting this PR; existing persisted payloads remain backward-compatible and unknown recovery fields are ignored by the prior runtime.

@numnx
numnx marked this pull request as ready for review July 14, 2026 10:14
@numnx
numnx enabled auto-merge July 14, 2026 10:17
@numnx
numnx disabled auto-merge July 14, 2026 10:41
@numnx
numnx enabled auto-merge July 14, 2026 10:41
@numnx
numnx merged commit aa32b60 into dev Jul 14, 2026
40 checks passed
@numnx
numnx deleted the fix/sprint-recovery-after-restart branch July 14, 2026 10:59
@numnx numnx mentioned this pull request Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant