Skip to content

fix(worker): fence work after queue claim loss - #301

Merged
Joncallim merged 28 commits into
codex/pr198-producer-5a2-r3b-claim-renewalfrom
codex/pr198-producer-5a2-r3b-fence
Jul 29, 2026
Merged

fix(worker): fence work after queue claim loss#301
Joncallim merged 28 commits into
codex/pr198-producer-5a2-r3b-claim-renewalfrom
codex/pr198-producer-5a2-r3b-fence

Conversation

@Joncallim

Copy link
Copy Markdown
Owner

Source Issue

Issue: #179

Summary

This stacked child makes definitive queue-claim loss an explicit cancellation and ownership fence. Once Redis reports stale_not_owner, the old worker aborts in-flight Architect generation and cannot persist task, attempt, artifact, question, work-package, checkpoint, event, or terminal queue mutations.

Key points:

  • adds a stable ClaimLeaseFence with a typed, non-disclosing loss error and AbortSignal;
  • preserves the parent contract that an initial transient Redis failure denies business startup without declaring permanent loss, while periodic transient failures remain nonfatal for the later bounded-outage slice;
  • carries the fence explicitly through task, answers, approval, Architect, workforce materialization, and handoff boundaries;
  • prevents broad catches, including startTaskAttempt, from misreporting lease loss as persistence/business failure;
  • keeps execution fail-closed and does not change retry/dead-letter receipt architecture.

Evidence

  • focused worker/orchestrator/handoff suites: 94 passed
  • full unit suite: 1,696 passed; existing integration-only skips unchanged
  • disposable Redis proof: 3/3 passed with QUEUE_OCCURRENCE_REDIS_MULTIPLICITY_OK, QUEUE_OCCURRENCE_REDIS_RECOVERY_OK, and QUEUE_OCCURRENCE_REDIS_SHUTDOWN_OK
  • ESLint: passed
  • TypeScript --noEmit: passed
  • git diff --check: passed
  • mutation checks: treating stale_not_owner as renewed failed two fence tests; removing the post-business ownership assertion failed the source contract test

The local Next build compiled successfully through webpack, then stopped on the parent branch's existing invalid Page export unresolvedRequiredFilesystemGrants in app/dashboard/tasks/[id]/page.tsx. That file is unchanged in this child. A standard Turbopack build could not run from the isolated worktree because its untracked dependency symlink points outside the Turbopack filesystem root.

Scope note

web/worker/workforce-materializer.ts is included because materialization performs awaited reads inside a transaction before writing packages and approval gates. The optional synchronous ownership callback makes those boundaries rollback safely without changing non-queue callers.

Remaining work

The bounded transient-renewal outage policy remains a separate R3C slice. This draft does not merge or resolve the parent review finding.

…ested-fence

fix(worker): fence nested handoff mutations after claim loss
@Joncallim
Joncallim marked this pull request as ready for review July 29, 2026 17:22
@Joncallim
Joncallim merged commit b20c23b into codex/pr198-producer-5a2-r3b-claim-renewal Jul 29, 2026
4 checks passed
@Joncallim
Joncallim deleted the codex/pr198-producer-5a2-r3b-fence branch August 2, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant