feat(state): prioritize durable cluster intake

[RomneyDa]

Summary

This is 2/5 in the cluster-fixer reliability stack, based on PR 881.

• Adds cluster_intake as a first-class durable state-append kind.
• Gives cluster intake bounded coordinator and queue priority so it cannot starve behind unrelated comment/review traffic.
• Preserves fairness by sharing a priority budget and yielding to ordinary writers.
• Carries coordinator class and queue metadata through the append client contract.
• Adds migration, fairness, lease, timeout, and priority regression coverage.

Validation

• pnpm run build:all
• 253 focused dashboard queue, append-client, and coordinator tests
• Included in the full stack's static, lint, and regression validation

Stack

1. Candidate selection and historical dedupe
2. This PR — durable intake queue priority
3. Durable intake intent contract
4. Exactly-once dispatch and recovery
5. Workflow cutover and result publication hardening

No timeout is enlarged, and no merge/automerge or production gate is enabled.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed July 27, 2026, 3:06 AM ET / 07:06 UTC.

ClawSweeper review

What this changes

The branch adds a durable cluster_intake state-append kind, gives it bounded shared priority in the state-writer coordinator, and adds migration, fairness, and intake-candidate regression coverage.

Merge readiness

⚠️ Ready for maintainer review - 4 items remain

Keep this PR open for maintainer review. It introduces a durable-state schema migration and shared writer scheduling policy in the middle of a five-layer cluster-intake stack; the supplied migration and fairness tests are meaningful, but landing should wait for an upgrade-path exercise against the stacked/current deployment shape. Likely related people: RomneyDa (recent area contributor, medium confidence) and steipete (recent reviewer of the rebased stack, low confidence).

Priority: P2
Reviewed head: 1b65ffd85f268f4fe61fe49aa18fce44a3e94d26
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	The patch has focused migration and fairness coverage, but the persisted-state upgrade and shared-writer behavior still need maintainer-visible operational validation.
Proof confidence	🌊 off-meta tidepool	Not applicable: The author is a repository member, so the external-contributor real-behavior-proof gate does not apply; the remaining need is maintainer-facing staged upgrade evidence for persistent-state behavior.
Patch quality	🦐 gold shrimp (3/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Not applicable	Not applicable: The author is a repository member, so the external-contributor real-behavior-proof gate does not apply; the remaining need is maintainer-facing staged upgrade evidence for persistent-state behavior.
Evidence reviewed	4 items	Durable append-window migration: The PR adds cluster_intake to the persisted append-kind constraint and normalizes retry-history columns before rebuilding older schemas, directly affecting upgrades that skipped intermediate deployments. Shared-priority coordinator policy: The coordinator adds cluster_intake as a writer class and tracks bounded consecutive priority turns, so the priority policy applies to durable state admission rather than only a caller-side queue. Migration and fairness coverage: The branch adds a regression test for retaining materialization retry history while migrating the append-window schema, plus tests for intake/publication priority sharing and ordinary-writer yielding.
Findings	None	None.
Security	None	None.

How this fits together

Cluster intake turns selected repair clusters into durable state before later layers publish and dispatch planning work. This PR changes the Durable Object append window and its shared writer coordinator so intake writes can make bounded progress without permanently starving ordinary state writers.

flowchart LR
  A[Cluster candidate selection] --> B[Cluster intake append]
  B --> C[Durable append window]
  C --> D[State writer coordinator]
  D --> E[Materialization and publication]
  E --> F[Planning job dispatch]
  D --> G[Ordinary state writers]

Loading

Decision needed

Question	Recommendation
Should the durable cluster_intake migration and shared writer-priority policy land now as the foundation for the remaining cluster-intake stack, after staged upgrade evidence is attached?	Require staged upgrade proof: Run the stacked/current-to-new schema upgrade with pre-existing queued rows and retry history, then capture bounded mixed-writer behavior before merging this foundation layer.

Why: This is a compatibility and availability decision over persisted Durable Object state and scheduling semantics; automated tests reduce risk but cannot establish the production upgrade and contention behavior by themselves.

Before merge

• Resolve merge risk (P1) - The append-window and coordinator migrations change persisted Durable Object table constraints; a deployment that skips intermediate versions still needs stacked upgrade evidence that queued rows and retry history survive intact.
• Resolve merge risk (P1) - The new bounded priority policy is covered by unit tests, but operational acceptance should confirm that cluster intake progresses under ordinary-writer load without increasing ordinary-write latency beyond the intended shared budget.
• Complete next step (P2) - A maintainer must decide whether the persistent-state migration and shared scheduling policy have enough staged upgrade evidence to serve as the base for the dependent intake stack.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Patch surface	15 files affected; 1,718 additions and 97 deletions	The change spans durable storage, scheduling, repair tooling, and regression tests, so stack-level upgrade review matters more than a narrow queue tweak.
Persisted scheduling paths	2 durable schema/coordinator modules changed	Both the append-window constraint and writer-class constraint participate in upgrade compatibility and availability behavior.

Merge-risk options

Maintainer options:

1. Prove the upgrade path before merge (recommended)
   Exercise an upgrade with queued append rows and retry-history fields present, then show that the new priority class remains bounded under mixed writer traffic.
2. Accept the operational migration risk
   Merge the unit-tested migration and schedule post-merge production acceptance for the complete five-layer stack.
3. Pause the reliability stack
   Hold this PR and its dependent layers if maintainers do not want to change the durable writer scheduling contract.

Technical review

Best possible solution:

Land this layer only after a staged upgrade from the current merged candidate-selection state proves preservation of queued append rows and retry history, then verify mixed ordinary, publication, and cluster-intake load before merging the dependent stack layers.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR introduces an internal durability and scheduling capability rather than fixing a separately reproducible user report. The relevant verification is an upgrade and mixed-load exercise against the changed Durable Object paths.

Is this the best way to solve the issue?

Unclear: the bounded shared-priority design matches the downstream durable-intake stack, but the best landing path still depends on maintainer acceptance of the persisted-state migration and operational scheduling contract.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d1c837ea2f63.

Labels

Label justifications:

• P2: This is a bounded reliability improvement with meaningful operator impact, but it does not show an active user-facing outage.
• merge-risk: 🚨 compatibility: The PR rebuilds persisted append-window and writer-class constraints, so older Durable Object state must remain readable through upgrade.
• merge-risk: 🚨 availability: It changes admission priority for a shared state writer, which can affect progress and wait time under concurrent load.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The author is a repository member, so the external-contributor real-behavior-proof gate does not apply; the remaining need is maintainer-facing staged upgrade evidence for persistent-state behavior.

Evidence

What I checked:

• Durable append-window migration: The PR adds cluster_intake to the persisted append-kind constraint and normalizes retry-history columns before rebuilding older schemas, directly affecting upgrades that skipped intermediate deployments. (dashboard/exact-review-queue.ts:5407, 1b65ffd85f26)
• Shared-priority coordinator policy: The coordinator adds cluster_intake as a writer class and tracks bounded consecutive priority turns, so the priority policy applies to durable state admission rather than only a caller-side queue. (dashboard/state-writer-coordinator.ts:92, 1b65ffd85f26)
• Migration and fairness coverage: The branch adds a regression test for retaining materialization retry history while migrating the append-window schema, plus tests for intake/publication priority sharing and ordinary-writer yielding. (test/dashboard-worker.test.ts:8286, 1b65ffd85f26)
• Stack provenance: The PR is explicitly layer 2 of the durable cluster-intake stack. Its predecessor is now merged into current main at d1c837ea2f63bcb36c11f9d962a539527760f969, while the next durable-intent and dispatch layers remain open and depend on this head. (d1c837ea2f63)

Likely related people:

• RomneyDa: Authored the two commits in this stacked reliability path, including the merged candidate-selection predecessor and this durable queue layer. (role: recent area contributor; confidence: medium; commits: 26cc500946d1, 1b65ffd85f26; files: dashboard/exact-review-queue.ts, dashboard/state-writer-coordinator.ts, src/repair/state-append-client.ts)
• steipete: Requested review and updated the base reference after the predecessor layer merged, making this person a likely routing point for the stacked landing decision. (role: recent reviewer; confidence: low; commits: d1c837ea2f63; files: dashboard/exact-review-queue.ts, dashboard/state-writer-coordinator.ts)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Attach a redacted staged upgrade transcript showing queued rows and retry-history fields before and after migration.
• Capture mixed-load evidence that verifies cluster intake gets bounded progress while ordinary writers still yield within the shared priority budget.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)

• reviewed 2026-07-27T04:56:45.127Z sha 1b65ffd :: needs maintainer review before merge. :: none

[clawsweeper]

ClawSweeper status: review started.

I am starting a fresh review of this pull request: feat(state): prioritize durable cluster intake This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.