Skip to content

feat(sync): reconcile generated delivery handoffs - #2898

Merged
stranske merged 3 commits into
mainfrom
codex/issue-2881-generated-pr-reconciliation
Aug 1, 2026
Merged

feat(sync): reconcile generated delivery handoffs#2898
stranske merged 3 commits into
mainfrom
codex/issue-2881-generated-pr-reconciliation

Conversation

@stranske

@stranske stranske commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Source: Issue #2881

Closes #2881

Automated Status Summary

Scope

Generated PR ownership is split across overlapping controllers. Maint 71 filters only sync/workflows- heads today (.github/workflows/maint-71-merge-sync-prs.yml:284), while Maint 82 and .github/scripts/sync_dependency_campaign.js:493-530 separately decide which exceptions are claimable by Codex. Dev-tool-sync PRs therefore follow a different merge path, and unchanged exception state can continue to produce automation work.

This is a current efficiency defect: repeated observation of the same blocker consumes agent runs without changing repository state. One repository-owned reconciliation contract should decide merge/close disposition, and agent handoff should occur only when the exception fingerprint changes.

Tasks

  • Extend .github/workflows/maint-71-merge-sync-prs.yml to reconcile both sync/workflows- and deps/sync-dev-versions- PRs through the same contract.
  • Centralize classification in .github/scripts/sync_pr_merge_contract.js: current, awaiting-checks, review-blocked, repo-local-failure, shared-source-failure, superseded, expired, and owner-decision.
  • Have Maint 71 emit a normalized result record containing PR identity, head SHA, delivery generation, check/review state, disposition, blocker owner, and exact next command.
  • Make .github/workflows/maint-82-sync-dependency-auto-pilot.yml consume those records and own durable campaign state rather than recomputing merge disposition independently.
  • Extend the existing fingerprint in .github/scripts/sync_dependency_campaign.js to include only state that should trigger new work; preserve updated_at, claim generation, and lease history.
  • Apply campaign:needs-local-codex only for new or materially changed actionable exception fingerprints, and remove it after terminal disposition or successful claim completion.
  • Document the remote handoff schema and local-consumer migration in docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md; mark local watcher changes as an operator follow-up, not repository implementation.
  • Add observability showing new, unchanged, resolved, and re-opened exception counts per run.

Acceptance criteria

  • .github/scripts/__tests__/sync_dependency_campaign.test.js proves the same exception observed on three consecutive runs creates one claim generation and one agent handoff.
  • .github/scripts/__tests__/sync_pr_merge_contract.test.js applies identical merge/review/check rules to sync and dev-tool-sync fixtures.
  • Workflow tests prove Maint 71 is the only path that merges or closes generated PRs and Maint 82 only persists state and requests bounded exception work.
  • A changed head SHA, review-thread set, check-failure cluster, or delivery generation creates a new fingerprint; a timestamp-only change does not.
  • Durable state records the exact blocker owner and next command, allowing an opener/closer run to resume without rediscovery.
  • Deliberate-break test: remove fingerprint persistence or vary only updated_at, verify the campaign test detects duplicate handoffs, then revert and verify it passes.
  • Run python scripts/dev_check.py --action test and workflow validation successfully.

Summary by CodeRabbit

  • New Features

    • Added support for classifying and managing generated delivery branches, including development-tool synchronization branches.
    • Merge processing now tracks review status, delivery outcomes, handoffs, and restart metadata across workflow results.
    • Active review threads can block merges until resolved.
    • Branch cleanup now covers all supported generated delivery branches.
  • Documentation

    • Documented generated delivery ownership, campaign coordination, delivery records, and operator handoffs.

@stranske stranske added agent:codex Agent-created issues from Codex agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation labels Aug 1, 2026
Copilot AI review requested due to automatic review settings August 1, 2026 23:11
@cursor

cursor Bot commented Aug 1, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@stranske stranske added the agent:retry Add to trigger agent retry after rate limit or pause label Aug 1, 2026
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 42 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3b4430df-4e97-4c01-85b5-04ecc9849d8f

📥 Commits

Reviewing files that changed from the base of the PR and between 692bb59 and 0024aef.

📒 Files selected for processing (6)
  • .github/scripts/__tests__/sync-pr-merge-contract.test.js
  • .github/scripts/sync_pr_merge_contract.js
  • .github/workflows/maint-71-merge-sync-prs.yml
  • docs/ops/CONSUMER_REPO_MAINTENANCE.md
  • docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md
  • tests/workflows/test_workflow_agents_consolidation.py
📝 Walkthrough

Walkthrough

The PR extends generated delivery handling to dev-tool sync branches. It centralizes classification, review blocking, cleanup, result metadata, and handoff records in Maint 71. Tests and operations documentation cover the updated contract.

Changes

Generated delivery reconciliation

Layer / File(s) Summary
Delivery classification and handoffs
.github/scripts/sync_dependency_campaign.js, .github/scripts/sync_pr_merge_contract.js
The contract recognizes sync and dev-tool sync branches, classifies delivery states, cleans up both branch types, and emits versioned handoff records.
Maint 71 reconciliation flow
.github/workflows/maint-71-merge-sync-prs.yml
The workflow evaluates review threads and delivery state once, blocks review-blocked PRs, and reuses shared metadata across result paths.
Classification and handoff tests
.github/scripts/__tests__/sync-pr-merge-contract.test.js
Tests cover both delivery lanes, trusted actors, review-blocked results, merge-report fields, and restart metadata.
Operational ownership documentation
docs/ops/CONSUMER_REPO_MAINTENANCE.md, docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md
Documentation defines generated delivery ownership, campaign records, dispositions, handoffs, and watcher responsibilities.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Maint71Workflow
  participant GeneratedPR
  participant DeliveryContract
  participant MergeReport
  Maint71Workflow->>GeneratedPR: read branch, review threads, and checks
  Maint71Workflow->>DeliveryContract: classify delivery state
  DeliveryContract-->>Maint71Workflow: disposition and next command
  Maint71Workflow->>MergeReport: record result and handoff
Loading

Possibly related issues

Possibly related PRs

  • stranske/Workflows#2886 — Extends the same sync merge contract and Maint 71 workflow with trusted-PR and review-thread classification.
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR centralizes both delivery lanes and handoffs [#2881], but omits Maint 82 durable-state consumption, fingerprint deduplication, observability, and required tests. Add the Maint 82 consumer and state changes, fingerprint deduplication, observability, and missing acceptance tests before merge.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Out of Scope Changes check ✅ Passed The code, tests, workflow updates, and documentation directly support the linked issue's reconciliation, handoff, ownership, and contract objectives.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the PR's main change: reconciling generated delivery handoffs for sync workflows.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/issue-2881-generated-pr-reconciliation

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Runner dispatch state for codex on PR #2898. Do not edit.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/scripts/sync_pr_merge_contract.js:
- Around line 70-71: Update the active review-thread decision in the
merge-contract logic to treat negative and non-finite activeReviewThreadCount
values as blocking evidence failures, rather than allowing the ready path to
return current. Use the review-query retry command for this blocker, preserve
the existing resolve-active-review-threads behavior for positive counts, and add
a regression test covering activeReviewThreadCount: -1.
- Around line 269-282: Update buildDeliveryHandoff so it only returns a handoff
when the PR result includes valid head_sha and delivery_generation values,
rather than emitting records with empty normalized restart data. Ensure all PR
result paths populate these delivery fields before calling buildDeliveryHandoff,
or reject and avoid publishing incomplete records for stale_closed,
stale_close_failed, and delivery_contract_blocked outcomes.

In @.github/workflows/maint-71-merge-sync-prs.yml:
- Around line 137-140: Update the workflow’s open-PR filtering to use
isTrustedGeneratedDeliveryPr instead of isTrustedSyncPr, allowing trusted
deps/sync-dev-versions- PRs to reach classifyGeneratedPr and the merge flow.
Adjust the active-PR selection logic to recognize the generated delivery lane
and avoid treating valid dev-tool PRs as missing or stale.

In `@docs/ops/CONSUMER_REPO_MAINTENANCE.md`:
- Around line 444-451: Update the “Generated delivery ownership” heading in the
document to use level-two Markdown syntax and ensure there is a blank line
before it, preserving the section’s existing content.

In `@docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md`:
- Around line 21-24: Update the campaign-state and local-watcher documentation
in SYNC_DEPENDENCY_CAMPAIGN.md so local agent work is requested only when an
actionable exception fingerprint materially changes. Add the actionable
predicate to the handoff condition and use identical wording wherever Maint 82
and watcher behavior is described.
- Around line 9-19: Update classifyGeneratedPr so the shared review-thread
predicate treats activeReviewThreadCount values indicating unknown state,
including -1 from pagination or API errors, as blocking alongside positive
counts. Apply this consistently to both sync/workflows-* and
deps/sync-dev-versions-* lanes, and add a regression test covering unavailable
review-thread evidence.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c61e1863-3df0-4c25-9f0d-220781167798

📥 Commits

Reviewing files that changed from the base of the PR and between d1e5020 and 692bb59.

📒 Files selected for processing (6)
  • .github/scripts/__tests__/sync-pr-merge-contract.test.js
  • .github/scripts/sync_dependency_campaign.js
  • .github/scripts/sync_pr_merge_contract.js
  • .github/workflows/maint-71-merge-sync-prs.yml
  • docs/ops/CONSUMER_REPO_MAINTENANCE.md
  • docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md

Comment thread .github/scripts/sync_pr_merge_contract.js Outdated
Comment thread .github/scripts/sync_pr_merge_contract.js
Comment thread .github/workflows/maint-71-merge-sync-prs.yml
Comment thread docs/ops/CONSUMER_REPO_MAINTENANCE.md Outdated
Comment thread docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md
Comment thread docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to unify the “generated delivery” reconciliation contract for both consumer-sync (sync/workflows-*) and dev-tool-sync (deps/sync-dev-versions-*) PRs by centralizing classification/handoff fields and documenting the intended ownership model (Maint 71 as the sole merge/close reconciler; Maint 82 as durable campaign state).

Changes:

  • Adds a documented contract for generated delivery handoffs and clarifies operator expectations in ops docs.
  • Extends .github/scripts/sync_pr_merge_contract.js with generated-delivery lane helpers, a classifyGeneratedPr disposition helper, and a machine-readable handoff_records surface.
  • Adds initial Maint 71 wiring to record delivery context fields and a review-blocked short-circuit, plus test coverage for new helpers.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md Introduces the intended campaign/lease contract and Maint 71/Maint 82 ownership model.
docs/ops/CONSUMER_REPO_MAINTENANCE.md Adds operator-facing guidance referencing the new campaign contract doc.
.github/workflows/maint-71-merge-sync-prs.yml Starts emitting normalized delivery context fields and review-blocked handling in Maint 71 results.
.github/scripts/sync_pr_merge_contract.js Adds generated-delivery lane detection, PR classification, and handoff record building.
.github/scripts/sync_dependency_campaign.js Expands sync PR detection to include dev-tool-sync branch prefix.
.github/scripts/tests/sync-pr-merge-contract.test.js Adds tests for lane detection, generated delivery classification, and handoff record shape.
Suppressed comments (3)

.github/workflows/maint-71-merge-sync-prs.yml:146

  • Maint 71 still filters open PRs with isTrustedSyncPr (sync/workflows-* only), and the selection helpers are still sync-specific (selectMergeEligibleSyncPr, syncBranchForHash). As a result, dev-tool-sync branches (deps/sync-dev-versions-*) will not be reconciled/merged/closed by this workflow despite the new delivery-lane plumbing.
            const {
              buildMarkdownSummary,
              buildMergeReport,
              classifyGeneratedPr,
              classifySyncPrChecks,
              collectDeletableSyncBranches,
              generatedDeliveryLane,
              normalizeSyncHash,
              parseBooleanInput,
              isTrustedSyncPr,
              selectMergeEligibleSyncPr,
              selectSyncPrGatingChecks,
            } = require('./.github/scripts/sync_pr_merge_contract.js');

docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md:12

  • This doc states Maint 71 applies the same merge/close reconciliation contract to deps/sync-dev-versions-* PRs, but maint-71-merge-sync-prs.yml still selects open PRs via isTrustedSyncPr (sync/workflows-* only). Either the workflow needs to be updated to actually reconcile dev-tool-sync PRs, or this doc should be softened until that support lands to avoid an operationally misleading contract.
Maint 71 is the sole merge/close reconciler. It applies the same contract to
both `sync/workflows-*` consumer-sync branches and
`deps/sync-dev-versions-*` shared dev-tool propagation branches:

.github/scripts/sync_dependency_campaign.js:100

  • isSyncPullRequest() now classifies deps/sync-dev-versions-* PRs as sync PRs for campaign purposes, but Maint 71 still only reconciles sync/workflows-* PRs. This mismatch can cause dev-tool-sync deliveries to enter campaign tracking without a corresponding merge/close authority, reintroducing repeated handoffs / non-terminal exception churn.
function isSyncPullRequest(pr = {}) {
  const headRef = cleanString(pr.head?.ref || pr.headRefName || pr.headRef || pr.head_ref);
  const title = cleanString(pr.title).toLowerCase();
  const body = cleanString(pr.body);
  const labels = labelsForPullRequest(pr).map((label) => label.toLowerCase());
  return (
    headRef.startsWith(SYNC_BRANCH_PREFIX) ||
    headRef.startsWith(DEV_TOOL_SYNC_BRANCH_PREFIX) ||
    title.startsWith('chore: sync workflow templates') ||
    labels.includes('sync') ||
    body.includes('workflows-sync-lifecycle')
  );

Comment thread .github/workflows/maint-71-merge-sync-prs.yml
Comment thread .github/scripts/sync_pr_merge_contract.js
Comment thread docs/ops/SYNC_DEPENDENCY_CAMPAIGN.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 692bb59633

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/scripts/sync_pr_merge_contract.js
Comment thread .github/scripts/sync_pr_merge_contract.js
Comment thread .github/scripts/sync_pr_merge_contract.js Outdated
Comment thread .github/scripts/sync_pr_merge_contract.js
@stranske
stranske temporarily deployed to agent-high-privilege August 1, 2026 23:27 — with GitHub Actions Inactive
@stranske

stranske commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

Addressed the current inline review set in 0c96e097 and 0024aef4: Maint 71 now selects all trusted generated-delivery lanes (including dev-tool sync), fails closed when review evidence is unavailable, protects active dev-tool branches during cleanup, omits incomplete restart handoffs, uses the precise delivery marker/actionable-fingerprint documentation, and directly closes expired or superseded leased attempts with normalized context. Validation: node --test .github/scripts/__tests__/sync-pr-merge-contract.test.js (20 passed); Python 3.12 workflow suites (73 passed); workflow YAML parse, actionlint, API-wrapper guard, and diff check passed.

@stranske
stranske temporarily deployed to agent-high-privilege August 1, 2026 23:29 — with GitHub Actions Inactive
@stranske
stranske merged commit f425c5c into main Aug 1, 2026
51 checks passed
@stranske
stranske deleted the codex/issue-2881-generated-pr-reconciliation branch August 1, 2026 23:38
@agents-workflows-bot

Copy link
Copy Markdown
Contributor

🤖 Bot Comment Handler

  • Agent: codex
  • Bot comments to address: 9

The agent has been assigned to this PR to address the bot review comments.

Instructions for agent

  1. Implement suggested fixes that improve the code
  2. Skip suggestions that don't apply (note why in your response)

The bot comment handler workflow has prepared context in the artifacts.

@stranske stranske added the verify:compare Compare multiple LLM evaluations label Aug 1, 2026
@stranske
stranske temporarily deployed to agent-high-privilege August 1, 2026 23:39 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Provider Comparison Report

Provider Summary

Provider Model Verdict Confidence Summary
openai gpt-5.6-terra CONCERNS 86% The PR appears to make a useful partial implementation: Maint 71 is expanded, the shared merge contract is extended, documentation and efficiency metrics are added, and a contract fixture test is i...
anthropic claude-sonnet-5 CONCERNS 68% The PR delivers meaningful pieces of the requested reconciliation work: it extends maint-71's workflow (+85/-41) and sync_pr_merge_contract.js (+81/-3) with additional classification logic, and add...
📋 Full Provider Details (click to expand)

openai

  • Model: gpt-5.6-terra
  • Verdict: CONCERNS
  • Confidence: 86%
  • Scores:
    • Correctness: 6.0/10
    • Completeness: 5.0/10
    • Quality: 7.0/10
    • Testing: 5.0/10
    • Risks: 6.0/10
  • Summary: The PR appears to make a useful partial implementation: Maint 71 is expanded, the shared merge contract is extended, documentation and efficiency metrics are added, and a contract fixture test is introduced. However, the core end-to-end ownership transfer is not demonstrated because Maint 82 is unchanged even though it is explicitly required to consume Maint 71 result records and cease independent disposition decisions. Several acceptance-specific campaign and workflow ownership tests are also absent from the changed test set. The implementation is therefore not sufficiently complete to verify the documented acceptance criteria.
  • Concerns:
    • Maint 82 (.github/workflows/maint-82-sync-dependency-auto-pilot.yml) is not changed in this PR, despite the central requirement that it consume Maint 71 normalized reconciliation records and stop independently recomputing merge disposition. Given the documented prior split ownership, this leaves the principal cross-controller reconciliation requirement unproven and likely incomplete.
    • The durable campaign/fingerprint implementation changes only by two lines in sync_dependency_campaign.js. The stated requirements include preserving durable fields, gating labels and handoffs on materially changed actionable fingerprints, removing labels after terminal/claim completion, and including head SHA, review-thread set, failure cluster, and delivery generation while excluding timestamp-only changes. The limited implementation delta and lack of corresponding test changes do not provide sufficient evidence that this full behavior is implemented.
    • The acceptance criterion requiring a campaign test demonstrating that the same exception on three consecutive runs produces exactly one claim generation and one handoff is not addressed by a changed campaign test file. Likewise, no visible test implements the required deliberate-break regression demonstration for removed fingerprint persistence or timestamp-only changes.
    • The new merge-contract test is small and appears focused on equivalent classification for the two PR head families, but the diff summary does not show tests for all required classification outcomes: current, awaiting-checks, review-blocked, repo-local-failure, shared-source-failure, superseded, expired, and owner-decision.
    • No workflow test changes are shown that directly establish the required ownership invariant: Maint 71 is the sole generated-PR merge/close path and Maint 82 only persists state and requests bounded exception work. Existing workflow-agent consolidation tests receive only a minimal change.
    • The added health/metrics workflow improves observability, but it is supplementary to—not a substitute for—the required reconciliation handoff and durable-state behavior. Its addition increases operational surface area without resolving the missing Maint 82 integration evidence.

anthropic

  • Model: claude-sonnet-5
  • Verdict: CONCERNS
  • Confidence: 68%
  • Scores:
    • Correctness: 6.0/10
    • Completeness: 4.0/10
    • Quality: 6.0/10
    • Testing: 4.0/10
    • Risks: 6.0/10
  • Summary: The PR delivers meaningful pieces of the requested reconciliation work: it extends maint-71's workflow (+85/-41) and sync_pr_merge_contract.js (+81/-3) with additional classification logic, and adds a new health-83 observability workflow plus a substantial dependency_sync_efficiency_metrics.py script with dedicated tests. However, several core acceptance criteria appear unaddressed in the diff: there is no modification to maint-82-sync-dependency-auto-pilot.yml (required to consume merge-contract records and own durable state instead of recomputing disposition), the sync_dependency_campaign.js fingerprint extension is only 2 lines (inconsistent with the stated scope of a substantive fingerprint change), and there's no evidence of the dedicated sync_dependency_campaign.test.js proving the 'three consecutive runs => one claim generation/handoff' behavior or the deliberate-break test. Given these gaps in central pieces of the plan (Maint 82 consumption and deeper fingerprint/test coverage), this PR is best characterized as a partial, though functional, implementation rather than full satisfaction of the documented acceptance criteria.
  • Concerns:
    • The task explicitly requires 'Make .github/workflows/maint-82-sync-dependency-auto-pilot.yml consume those records and own durable campaign state rather than recomputing merge disposition independently,' but this workflow file is absent from the diff entirely. This is a core acceptance criterion (Maint 82 only persisting state, not recomputing disposition) that appears unaddressed.
    • sync_dependency_campaign.js only receives a 2-line change, which is inconsistent with the stated requirement to 'extend the existing fingerprint to include only state that should trigger new work; preserve updated_at, claim generation, and lease history.' This looks like a token/stub change rather than a substantive fingerprint extension.
    • The acceptance criteria explicitly names '.github/scripts/tests/sync_dependency_campaign.test.js' as proving that the same exception observed on three consecutive runs creates one claim generation and one agent handoff, but this test file is not present in the diff's changed-file list, meaning the specific behavior required by acceptance criteria is not demonstrably tested by this PR.
    • No evidence in the diff of the 'deliberate-break test' (removing fingerprint persistence or varying only updated_at to prove duplicate-handoff detection then reverting) that the acceptance criteria calls for.
    • The new test file is named 'sync-pr-merge-contract.test.js' (hyphenated) rather than 'sync_pr_merge_contract.test.js' (underscored) as specified in the acceptance criteria — possibly a naming mismatch or duplicate test file, unclear whether this is intentional or an inconsistency with the existing test suite convention.
    • New docs (SYNC_DEPENDENCY_CAMPAIGN.md, DEPENDENCY_SYNC_EFFICIENCY.md) are quite short (24 and 51 lines respectively) for documenting a 'remote handoff schema and local-consumer migration' as required — may be underspecified relative to the stated documentation task.
    • A large portion of the diff (health-83 workflow + dependency_sync_efficiency_metrics.py + its tests, totaling ~770 lines) is new observability tooling, which is valuable but seems to substitute for, rather than complete, the core reconciliation contract changes (Maint 82 integration, deeper fingerprint changes) called out in the tasks.

Agreement

  • Verdict: CONCERNS (all providers)
  • Correctness: scores within 1 point (avg 6.0/10, range 6.0-6.0)
  • Completeness: scores within 1 point (avg 4.5/10, range 4.0-5.0)
  • Quality: scores within 1 point (avg 6.5/10, range 6.0-7.0)
  • Testing: scores within 1 point (avg 4.5/10, range 4.0-5.0)
  • Risks: scores within 1 point (avg 6.0/10, range 6.0-6.0)

Disagreement

No major disagreements detected.

Unique Insights

  • openai: Maint 82 (.github/workflows/maint-82-sync-dependency-auto-pilot.yml) is not changed in this PR, despite the central requirement that it consume Maint 71 normalized reconciliation records and stop independently recomputing merge disposition. Given the documented prior split ownership, this leaves the principal cross-controller reconciliation requirement unproven and likely incomplete.; The durable campaign/fingerprint implementation changes only by two lines in sync_dependency_campaign.js. The stated requirements include preserving durable fields, gating labels and handoffs on materially changed actionable fingerprints, removing labels after terminal/claim completion, and including head SHA, review-thread set, failure cluster, and delivery generation while excluding timestamp-only changes. The limited implementation delta and lack of corresponding test changes do not provide sufficient evidence that this full behavior is implemented.; The acceptance criterion requiring a campaign test demonstrating that the same exception on three consecutive runs produces exactly one claim generation and one handoff is not addressed by a changed campaign test file. Likewise, no visible test implements the required deliberate-break regression demonstration for removed fingerprint persistence or timestamp-only changes.; The new merge-contract test is small and appears focused on equivalent classification for the two PR head families, but the diff summary does not show tests for all required classification outcomes: current, awaiting-checks, review-blocked, repo-local-failure, shared-source-failure, superseded, expired, and owner-decision.; No workflow test changes are shown that directly establish the required ownership invariant: Maint 71 is the sole generated-PR merge/close path and Maint 82 only persists state and requests bounded exception work. Existing workflow-agent consolidation tests receive only a minimal change.; The added health/metrics workflow improves observability, but it is supplementary to—not a substitute for—the required reconciliation handoff and durable-state behavior. Its addition increases operational surface area without resolving the missing Maint 82 integration evidence.
  • anthropic: The task explicitly requires 'Make .github/workflows/maint-82-sync-dependency-auto-pilot.yml consume those records and own durable campaign state rather than recomputing merge disposition independently,' but this workflow file is absent from the diff entirely. This is a core acceptance criterion (Maint 82 only persisting state, not recomputing disposition) that appears unaddressed.; sync_dependency_campaign.js only receives a 2-line change, which is inconsistent with the stated requirement to 'extend the existing fingerprint to include only state that should trigger new work; preserve updated_at, claim generation, and lease history.' This looks like a token/stub change rather than a substantive fingerprint extension.; The acceptance criteria explicitly names '.github/scripts/tests/sync_dependency_campaign.test.js' as proving that the same exception observed on three consecutive runs creates one claim generation and one agent handoff, but this test file is not present in the diff's changed-file list, meaning the specific behavior required by acceptance criteria is not demonstrably tested by this PR.; No evidence in the diff of the 'deliberate-break test' (removing fingerprint persistence or varying only updated_at to prove duplicate-handoff detection then reverting) that the acceptance criteria calls for.; The new test file is named 'sync-pr-merge-contract.test.js' (hyphenated) rather than 'sync_pr_merge_contract.test.js' (underscored) as specified in the acceptance criteria — possibly a naming mismatch or duplicate test file, unclear whether this is intentional or an inconsistency with the existing test suite convention.; New docs (SYNC_DEPENDENCY_CAMPAIGN.md, DEPENDENCY_SYNC_EFFICIENCY.md) are quite short (24 and 51 lines respectively) for documenting a 'remote handoff schema and local-consumer migration' as required — may be underspecified relative to the stated documentation task.; A large portion of the diff (health-83 workflow + dependency_sync_efficiency_metrics.py + its tests, totaling ~770 lines) is new observability tooling, which is valuable but seems to substitute for, rather than complete, the core reconciliation contract changes (Maint 82 integration, deeper fingerprint changes) called out in the tasks.

🔍 LangSmith Traces

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:codex Agent-created issues from Codex agent:retry Add to trigger agent retry after rate limit or pause agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation verify:compare Compare multiple LLM evaluations verify:create-new-pr

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Dependency/Sync] Consolidate generated-PR reconciliation and exception handoff

3 participants