Fix automerge terminal telemetry convergence

[hxy91819]

Root cause evidence

• Consolidate automerge worker health #648 merged at 2026-07-17T13:39:18Z through https://github.com/openclaw/clawsweeper/actions/runs/29584167178.
• That run's report and immutable command action ledger record the merge mutation and terminal command completion. The same router batch contained the original automerge activation, so the terminal: merged event generation conditions were satisfied.
• The run had the status ingest token configured, but the production metrics API retained only the activation for that session. The durable receiver accepts, atomically stores, deduplicates, and aggregates terminal events, so the missing boundary is the unacknowledged best-effort delivery path rather than automerge control behavior or aggregation.

What changed

• Add a separate five-minute Product Health reconciliation step to the existing dashboard telemetry workflow.
• Read only durable active sessions from the last 24 hours, oldest first, with an eight-session default and hard maximum of 20.
• Perform at most one authoritative GitHub PR read per candidate under a shared 15-second deadline (hard maximum 30 seconds).
• Emit a stable, retry-safe terminal event. merged_at always takes precedence over generic closed_at, and the existing first-terminal immutable projection remains unchanged.
• Keep reconciliation best-effort and continue-on-error, so GitHub, status API, or ingest failures cannot affect automerge.
• Rename dashboard coverage to Time-window coverage, show active-session count explicitly, and render No terminal samples yet instead of zero-like KPI cards when the denominator is empty.

Why reconciliation instead of an outbox

The durable active-session ledger already identifies sessions whose terminal delivery did not converge. Rechecking bounded authoritative PR status repairs both the existing missing terminal for #648 and future transport/timeout/response-loss cases. A new outbox would protect only failures after deployment and would still require a second backfill mechanism for already-active merged sessions.

This does not perform broad history backfill: candidates must have activity within 24 hours, the API response and GitHub reads are count-bounded, and open PRs remain active.

Validation

• node --test test/automerge-metrics.test.ts — 14 passed
• pnpm run build:dashboard — passed
• node --test test/dashboard-worker.test.ts — 136 passed
• pnpm run check — passed on Node v24.15.0
• $AUTOREVIEW --mode local --stream-engine-output — clean, no accepted/actionable findings; patch is correct (confidence 0.86)

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed July 17, 2026, 12:14 PM ET / 16:14 UTC.

Summary
The branch adds a bounded scheduled reconciliation job for missing automerge terminal events, exposes active-session queries, updates empty-sample dashboard presentation, and adds focused tests.

Reproducibility: yes. The merge of #648 is a concrete current-main production case where durable command records show terminal completion but the metrics service retained only activation.

Review metrics: 3 noteworthy metrics.

• Patch surface: 6 files; 387 added, 9 removed. The change is substantial but remains confined to dashboard telemetry, one workflow, and focused tests.
• Reconciliation bounds: 8 default / 20 maximum candidates; 15s default / 30s maximum. These caps limit scheduled API fan-out and workflow runtime exposure.
• Reported focused tests: 150 passed across 2 suites. The PR reports 14 automerge-metrics and 136 dashboard-worker tests passing in addition to the repository check.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• After deployment, capture redacted workflow or live status output showing the known active session receives exactly one merged terminal event.

Risk before merge

• [P1] The new scheduled path has strong unit coverage but no after-deployment observation yet showing that an already-active production session converges to its terminal event; a defect would leave dashboard telemetry stale, although continue-on-error isolates automerge itself.

Maintainer options:

1. Land with convergence monitoring (recommended)
   Merge after normal review, then verify that the known active session receives one stable terminal event and the scheduled job remains isolated from automerge.
2. Require pre-merge live output
   Keep the draft open until a manual run against a safe endpoint shows candidate discovery, authoritative PR lookup, and idempotent terminal ingestion.

Next step before merge

• [P2] No concrete code defect requires an automated repair; maintainers should complete normal draft review and decide whether the bounded production-risk evidence is sufficient.

Security
Cleared: The patch adds no dependency, third-party action, permission, secret exposure, or unbounded execution path; its token-bearing requests remain bounded and use existing configured endpoints.

Review details

Best possible solution:

Operate the bounded reconciliation as an isolated telemetry repair and confirm in production that the existing #648 session converges exactly once without affecting automerge execution.

Do we have a high-confidence way to reproduce the issue?

Yes. The merge of #648 is a concrete current-main production case where durable command records show terminal completion but the metrics service retained only activation.

Is this the best way to solve the issue?

Yes, provisionally. Bounded reconciliation is narrower than introducing a second durable outbox and can repair both the existing missed event and future ambiguous delivery failures, while production convergence still needs observation.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against ea22ada84dd0.

Label changes

Label changes:

• add P2: The bug produces inaccurate operational product-health telemetry but does not block or alter automerge execution.
• add merge-risk: 🚨 automation: The PR adds a recurring workflow step that reads GitHub state and writes production telemetry, a runtime automation path that unit tests cannot fully validate.
• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate does not apply to this MEMBER-authored PR; the body provides production root-cause evidence and test output, but not an after-deployment convergence observation.

Label justifications:

• P2: The bug produces inaccurate operational product-health telemetry but does not block or alter automerge execution.
• merge-risk: 🚨 automation: The PR adds a recurring workflow step that reads GitHub state and writes production telemetry, a runtime automation path that unit tests cannot fully validate.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate does not apply to this MEMBER-authored PR; the body provides production root-cause evidence and test output, but not an after-deployment convergence observation.

Evidence reviewed

What I checked:

• Observed current-main failure: The PR body ties the missing terminal event to the merge of Consolidate automerge worker health #648: the immutable action ledger recorded terminal completion, while the production metrics API retained only the activation. (ea22ada84dd0)
• Failure isolation: The new dashboard workflow step is explicitly best-effort and continue-on-error, preventing reconciliation failures from changing automerge control behavior. (.github/workflows/dashboard-ci.yml:34, 8e100b4b5819)
• Bounded production work: Reconciliation defaults to eight candidates and a shared 15-second deadline, with hard limits of 20 candidates and 30 seconds. (scripts/dashboard-reconcile-automerge.ts:7, 8e100b4b5819)
• Deterministic candidate selection: The metrics query supports active-only retrieval, orders those sessions oldest first to avoid starvation, and applies a bounded result limit. (dashboard/automerge-metrics.ts:176, 8e100b4b5819)
• Retry and precedence coverage: Focused tests exercise retry-safe terminal delivery for the observed merged session and verify merged status takes precedence over generic closure. (test/automerge-metrics.test.ts:270, 8e100b4b5819)
• Feature-history provenance: The affected dashboard surface was modified immediately beforehand by the merged Consolidate automerge worker health #648, whose head was 0e00da7 and whose merge produced current main ea22ada. (dashboard/worker.ts:9295, ea22ada84dd0)

Likely related people:

• hxy91819: Authored the merged dashboard consolidation immediately preceding this PR and the current reconciliation patch, giving them direct recent history across the affected workflow, metrics, dashboard, and tests. (role: recent area contributor; confidence: high; commits: 0e00da738bd5, 8e100b4b5819; files: .github/workflows/dashboard-ci.yml, dashboard/automerge-metrics.ts, dashboard/worker.ts)

What the crustacean ranks mean

• 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
• 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
• 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
• 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
• 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
• 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
• 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.