Skip to content

feat: compile consumer sync into one typed normalized plan - #2756

Merged
stranske merged 11 commits into
mainfrom
codex/issue-2751-typed-consumer-sync-plan
Jul 11, 2026
Merged

feat: compile consumer sync into one typed normalized plan#2756
stranske merged 11 commits into
mainfrom
codex/issue-2751-typed-consumer-sync-plan

Conversation

@stranske

@stranske stranske commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Source: Issue #2751

Closes #2751

Automated Status Summary

Scope

  • Add a typed Python manifest compiler for .github/sync-manifest.yml.
  • Normalize every sync entry into explicit records containing section, source, resolved local source, target, sync mode, directory flag, skip rules, overwrite rules, and stable content/effect fingerprints.
  • Resolve source ownership once using the documented template-vs-root section policy.
  • Emit JSON suitable for shell/workflow consumers.
  • Fail early on missing/ambiguous sources, duplicate effective targets, malformed skip rules, unsafe paths, and unsupported modes.
  • Migrate the maintenance sync, drift, template-hash, and manifest/template validation paths to consume the compiler or its normalized output.
  • Keep removal entries typed and distinct from file-copy entries.
  • Update the defining docs and consumer-template contract where behavior changes.

Context for Agent

Related Issues/PRs

Tasks

  • Implement the compiler and CLI with deterministic output.
  • Add unit fixtures for template-root precedence, repository-root precedence, directories, create-only/overwrite behavior, removals, duplicate targets, missing sources, and unsafe paths.
  • Replace duplicated resolution logic in covered Python validators.
  • Make the sync workflow prepare phase compile and validate the manifest before fan-out.
  • Make template hashing derive from the normalized plan.
  • Document the normalized plan contract and migration boundary.

Acceptance criteria

  • The same manifest entry resolves to the same source and target in sync, drift, hash, and validation paths.
  • Every declared copy entry is deliverable or validation fails before consumer mutation.
  • Duplicate effective targets and ambiguous source candidates are rejected with actionable diagnostics.
  • JSON output is deterministic for identical inputs and includes a schema version.
  • Existing intentional consumer skips and create-only exceptions remain intact.
  • No consumer write begins when compilation fails.

Summary by CodeRabbit

  • New Features
    • Added a deterministic, schema-validated consumer sync manifest compiler that produces a standardized sync plan with stable plan IDs and fingerprints.
    • Introduced GraphQL-based pull request context helpers and GitHub API caching utilities for PR-related workflows.
  • Bug Fixes
    • Updated sync, template validation, and drift checks to rely on the compiled plan for more consistent behavior.
    • Strengthened manifest/path safety, normalization, and duplicate-target detection; improved deterministic directory hashing.
  • Documentation
    • Documented the manifest schema, compilation, and sync plan validation process.
  • Tests
    • Expanded compiler, validation, determinism, and drift-check test coverage.

Copilot AI review requested due to automatic review settings July 11, 2026 00:31
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR adds a typed manifest compiler and deterministic consumer sync plan, migrates workflows and validators to the plan, adds schema and tests, updates synced PR-context scripts, and documents the compiler contract.

Changes

Consumer sync plan

Layer / File(s) Summary
Compiler contract and implementation
docs/contracts/schemas/..., scripts/sync_manifest_compiler.py
Defines typed entries and removals, source resolution, safety checks, deterministic hashes, validation, duplicate detection, and CLI output.
Workflow plan consumption
.github/workflows/..., .github/sync-manifest.yml
Compiles the manifest before validation and sync, consumes unified entries and removals, resolves compiled sources, and stages copy targets.
Validator migration
scripts/check_consumer_sync_drift.py, scripts/validate_template_sync.py, tests/scripts/...
Migrates drift and template validation to compiled manifests and typed policies, with updated tests.
Compiler and validator coverage
tests/scripts/test_sync_manifest_compiler.py
Tests source precedence, path and symlink safety, normalization, fingerprints, duplicate targets, deterministic CLI output, real-manifest compilation, and workflow consumption.
Manifest maintenance documentation
docs/ops/CONSUMER_REPO_MAINTENANCE.md
Documents typed skip rules, compiler validation, normalized plan fields, hashes, and invocation points.
Run metadata
langsmith-fleet-worker-attempt.json
Updates emission metadata, pull request number, selected model, and selection reason.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Manifest as sync-manifest.yml
  participant Compiler as sync_manifest_compiler.py
  participant Validator as check_consumer_sync_drift.py
  participant Workflow as maint-68-sync-consumer-repos.yml
  participant ConsumerRepo as Consumer repository
  Manifest->>Compiler: compile and validate YAML
  Compiler-->>Validator: typed manifest and removals
  Compiler-->>Workflow: deterministic consumer-sync-plan/v1
  Workflow->>ConsumerRepo: copy resolved sources to targets
  Validator->>ConsumerRepo: compare compiled targets with remote files
Loading

Possibly related PRs

  • stranske/Workflows#2496: Refactors the typed sync-manifest compilation pipeline that processes manifest entries and validates synchronized files.
  • stranske/Workflows#2706: Updates the typed sync-manifest compiler and consumer sync workflow for per-entry synchronization, including git_config entries.

Suggested labels: sync, automation, maintenance, github:actions

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The changeset includes an unrelated update to langsmith-fleet-worker-attempt.json run metadata, which is outside #2751's compiler and sync scope. Remove or split out the metadata-only file change into a separate PR unless it is required for the compiler work.
Docstring Coverage ⚠️ Warning Docstring coverage is 16.92% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: compiling consumer sync into a typed normalized plan.
Linked Issues check ✅ Passed The PR implements the typed compiler, migrates sync/drift/validation paths, adds schema/docs/tests, and preserves removals, skips, and create-only behavior required by #2751.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/issue-2751-typed-consumer-sync-plan

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR centralizes consumer sync-manifest interpretation by introducing a typed Python compiler that normalizes .github/sync-manifest.yml into a deterministic workflows.consumer-sync-plan/v1 JSON plan, then migrates sync/drift/validation surfaces to consume that normalized output so precedence and deliverability checks are consistent.

Changes:

  • Add scripts/sync_manifest_compiler.py to compile/validate the manifest into a typed, deterministic plan with stable hashes/fingerprints and early failure on invalid entries.
  • Update Maint 68 sync and drift/template-sync validators to use the compiled plan (single source of truth for resolved sources/targets and template hash).
  • Add schema + unit tests validating precedence, safety checks, duplicates, deterministic output, and real-manifest compilation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/scripts/test_validate_template_sync.py Updates test harness to copy the new compiler dependency alongside the validator.
tests/scripts/test_sync_manifest_compiler.py Adds unit coverage for compiler behavior, determinism, CLI outputs, and real-manifest validation against JSON schema.
tests/scripts/test_check_consumer_sync_drift.py Refactors drift tests to use typed manifest entries/compiled manifest instead of untyped dicts/YAML parsing.
scripts/validate_template_sync.py Switches template-sync validation to compile the manifest first and operate over typed entries.
scripts/sync_manifest_compiler.py Introduces the typed manifest compiler + deterministic plan/CLI + safety/uniqueness validation.
scripts/check_consumer_sync_drift.py Migrates drift checker to consume the compiled manifest and canonical source resolution.
docs/ops/CONSUMER_REPO_MAINTENANCE.md Documents the new compiler/plan contract, fields, determinism, and validation workflow integration.
docs/contracts/schemas/consumer-sync-plan-v1.schema.json Adds the machine-readable JSON schema for the normalized plan.
.github/workflows/maint-68-sync-consumer-repos.yml Compiles the manifest into manifest.json, uses normalized resolved_source/target, and derives template hash from plan output.
.github/workflows/health-70-validate-sync-manifest.yml Adds an explicit “compile to plan JSON” validation step for PRs touching the manifest.
.github/sync-manifest.yml Removes a duplicate .github/scripts/agents-guard.js entry from the scripts: section (now enforced by duplicate-target checks).

Comment thread scripts/check_consumer_sync_drift.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c17f04c994

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/maint-68-sync-consumer-repos.yml
@stranske-keepalive

stranske-keepalive Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Automated Status Summary

Head SHA: 4c94048
Latest Runs: ⏳ pending — Gate
Required contexts: Gate / gate, Health 45 Agents Guard / guard
Required: core tests (3.12): ⏳ pending, core tests (3.13): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job Result Logs
(no jobs reported) ⏳ pending

Coverage Overview

  • Coverage history entries: 1

Coverage Trend

Metric Value
Current 75.67%
Baseline 85.00%
Delta -9.33%
Minimum 70.00%
Status ✅ Pass

Top Coverage Hotspots (lowest coverage)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/validate_template_sync.py 0.0% 81
scripts/langchain/topic_splitter.py 19.1% 57
tools/codex_log_analyzer.py 19.6% 140
scripts/repo_review_round2_runner.py 25.1% 344
scripts/prune_agent_stubs.py 39.7% 26
scripts/repo_review_round1_runner.py 40.7% 133
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
tools/ci_failure_triage.py 44.2% 123
scripts/repo_review_backlog_scan.py 45.3% 116
scripts/repo_review_body_writer.py 46.5% 86
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58

Low Coverage Files (<50.0%)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/validate_template_sync.py 0.0% 81
scripts/langchain/topic_splitter.py 19.1% 57
tools/codex_log_analyzer.py 19.6% 140
scripts/repo_review_round2_runner.py 25.1% 344
scripts/prune_agent_stubs.py 39.7% 26
scripts/repo_review_round1_runner.py 40.7% 133
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
tools/ci_failure_triage.py 44.2% 123
scripts/repo_review_backlog_scan.py 45.3% 116
scripts/repo_review_body_writer.py 46.5% 86
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58

Updated automatically; will refresh on subsequent CI/Docker completions.


Keepalive checklist

Scope

  • Add a typed Python manifest compiler for .github/sync-manifest.yml.
  • Normalize every sync entry into explicit records containing section, source, resolved local source, target, sync mode, directory flag, skip rules, overwrite rules, and stable content/effect fingerprints.
  • Resolve source ownership once using the documented template-vs-root section policy.
  • Emit JSON suitable for shell/workflow consumers.
  • Fail early on missing/ambiguous sources, duplicate effective targets, malformed skip rules, unsafe paths, and unsupported modes.
  • Migrate the maintenance sync, drift, template-hash, and manifest/template validation paths to consume the compiler or its normalized output.
  • Keep removal entries typed and distinct from file-copy entries.
  • Update the defining docs and consumer-template contract where behavior changes.

Context for Agent

Related Issues/PRs

Tasks

  • Implement the compiler and CLI with deterministic output.
  • Add unit fixtures for template-root precedence, repository-root precedence, directories, create-only/overwrite behavior, removals, duplicate targets, missing sources, and unsafe paths.
  • Replace duplicated resolution logic in covered Python validators.
  • Make the sync workflow prepare phase compile and validate the manifest before fan-out.
  • Make template hashing derive from the normalized plan.
  • Document the normalized plan contract and migration boundary.

Acceptance criteria

  • The same manifest entry resolves to the same source and target in sync, drift, hash, and validation paths.
  • Every declared copy entry is deliverable or validation fails before consumer mutation.
  • Duplicate effective targets and ambiguous source candidates are rejected with actionable diagnostics.
  • JSON output is deterministic for identical inputs and includes a schema version.
  • Existing intentional consumer skips and create-only exceptions remain intact.
  • No consumer write begins when compilation fails.

@stranske
stranske temporarily deployed to agent-high-privilege July 11, 2026 00:41 — with GitHub Actions Inactive
@stranske stranske added agent:codex Agent-created issues from Codex agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation agent:retry Add to trigger agent retry after rate limit or pause labels Jul 11, 2026
@stranske
stranske temporarily deployed to agent-standard July 11, 2026 01:05 — with GitHub Actions Inactive
@stranske

stranske commented Jul 11, 2026

Copy link
Copy Markdown
Owner Author

Runner dispatch state for autofix on PR #2756. Do not edit.

@stranske
stranske temporarily deployed to agent-standard July 11, 2026 01:06 — with GitHub Actions Inactive
@stranske
stranske temporarily deployed to agent-high-privilege July 11, 2026 01:06 — with GitHub Actions Inactive
@stranske-keepalive

stranske-keepalive Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

🤖 Keepalive Loop Status

PR #2756 | Agent: Codex | Iteration 0/12

Current State

Metric Value
Iteration progress [----------] 0/12
Action skip (needs-human)
Disposition skipped
Gate success
Tasks 0/12 complete
Timeout 45 min (default)
Timeout usage 11m elapsed (26%, 34m remaining)
Keepalive ✅ enabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | unknown |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

⚠️ Failure Tracking

| Consecutive failures | 6/3 |
| Reason | agent-run-failed |

@stranske-keepalive

stranske-keepalive Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor
Keepalive Work Log (click to expand)
# Time (UTC) Agent Action Result Files Tasks Progress Commit Gate
0 2026-07-11 01:09:03 Codex run (agent-run-failed) retry failure 4 file(s) 0 0/12 fcd6d60
0 2026-07-11 01:12:09 Codex run (agent-run-failed) failure 4 file(s) 0 0/12 4a068b4 cancelled
0 2026-07-11 01:15:12 Codex run (agent-run-failed-repeat) failure 4 file(s) 0 0/12 7467a76 cancelled
0 2026-07-11 01:15:54 Codex skip (needs-human) skipped 0 0/12
0 2026-07-11 01:16:31 Codex skip (needs-human) skipped 0 0/12 cancelled
0 2026-07-11 01:19:44 Codex skip (needs-human) skipped 0 0/12 success
0 2026-07-11 01:23:42 Codex skip (needs-human) skipped 0 0/12 success
0 2026-07-11 01:30:05 Codex skip (needs-human) skipped 0 0/12 success
0 2026-07-11 01:34:31 Codex wait (missing-agent-label-transient) skipped 0 0/12 cancelled
0 2026-07-11 01:38:09 Codex wait (missing-agent-label-transient) skipped 0 0/12 success
0 2026-07-11 02:05:39 Codex run (agent-run-failed) retry failure 4 file(s) 0 0/12 4dd6845
0 2026-07-11 02:08:40 Codex run (agent-run-failed) retry failure 4 file(s) 0 0/12 321689c cancelled
0 2026-07-11 02:11:51 Codex run (agent-run-failed-repeat) retry failure 4 file(s) 0 0/12 beaac60 cancelled
0 2026-07-11 02:12:28 Codex skip (needs-human) retry skipped 0 0/12
0 2026-07-11 02:13:12 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 02:13:55 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 02:14:58 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 02:18:42 Codex skip (needs-human) retry skipped 0 0/12 success
0 2026-07-11 03:13:25 Codex fix (agent-run-failed-repeat) retry failure 4 file(s) 0 0/12 1dfa619 failure
0 2026-07-11 03:14:06 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 03:14:36 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 03:15:28 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 03:19:07 Codex skip (needs-human) retry skipped 0 0/12 failure
0 2026-07-11 03:35:20 Codex run (agent-run-failed-repeat) retry failure 4 file(s) 0 0/12 3aac21b success
0 2026-07-11 03:35:56 Codex skip (needs-human) retry skipped 0 0/12
0 2026-07-11 03:36:46 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 03:40:12 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 03:44:11 Codex skip (needs-human) retry skipped 0 0/12 success
0 2026-07-11 04:23:59 Codex run (agent-run-failed-repeat) retry failure 1 file(s) 0 0/12 c89671b
0 2026-07-11 04:24:42 Codex skip (needs-human) skipped 0 0/12 cancelled
0 2026-07-11 04:24:48 Codex skip (needs-human) retry skipped 0 0/12
0 2026-07-11 04:25:27 Codex skip (needs-human) retry skipped 0 0/12 cancelled
0 2026-07-11 04:25:35 Codex skip (needs-human) skipped 0 0/12 cancelled
0 2026-07-11 04:35:39 Codex skip (needs-human) skipped 0 0/12 success

@stranske stranske removed agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation agent:retry Add to trigger agent retry after rate limit or pause labels Jul 11, 2026
@agents-workflows-bot
agents-workflows-bot Bot temporarily deployed to agent-high-privilege July 11, 2026 01:10 Inactive
@agents-workflows-bot
agents-workflows-bot Bot temporarily deployed to agent-high-privilege July 11, 2026 01:13 Inactive
@agents-workflows-bot agents-workflows-bot Bot added the agent:needs-attention Agent needs human review or intervention label Jul 11, 2026
@stranske

Copy link
Copy Markdown
Owner Author

Resolved the remaining cyclic-symlink review finding in 533f5e6a: manifest compilation now converts RuntimeError from cyclic symlink resolution into a validation error, and adds a nested-loop regression test. Validation passed: uv run pytest tests/scripts/test_sync_manifest_compiler.py -q (30 passed), Ruff, Black check, py_compile, and git diff --check.

@stranske stranske removed agent:needs-attention Agent needs human review or intervention needs-human Requires human intervention or review labels Jul 11, 2026
stranske and others added 6 commits July 10, 2026 22:04
…ion paths

Introduces scripts/sync_manifest_compiler.py — a single, typed, deterministic
parser for .github/sync-manifest.yml.  Every caller that previously did raw
yaml.safe_load() + dict.get() guessing now uses compile_manifest(), which
validates all entries and raises ManifestCompileError before any consumer
mutation can happen.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@stranske
stranske force-pushed the codex/issue-2751-typed-consumer-sync-plan branch from 533f5e6 to e788692 Compare July 11, 2026 03:05
@stranske
stranske temporarily deployed to agent-high-privilege July 11, 2026 03:06 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Runner dispatch state for codex on PR #2756. Do not edit.

@agents-workflows-bot
agents-workflows-bot Bot temporarily deployed to agent-high-privilege July 11, 2026 03:13 Inactive
@stranske-keepalive stranske-keepalive Bot added agent:needs-attention Agent needs human review or intervention needs-human Requires human intervention or review labels Jul 11, 2026
@agents-workflows-bot
agents-workflows-bot Bot temporarily deployed to agent-high-privilege July 11, 2026 03:15 Inactive
@stranske
stranske temporarily deployed to agent-high-privilege July 11, 2026 03:26 — with GitHub Actions Inactive
@stranske stranske removed needs-human Requires human intervention or review agent:needs-attention Agent needs human review or intervention labels Jul 11, 2026
@stranske

Copy link
Copy Markdown
Owner Author

Closer recovery: fixed the Python 3.13-only cyclic-symlink gap in sync_manifest_compiler.py by using strict OS-level symlink resolution. Pushed c3f00477. Validation passed: Python 3.13 compiler suite (30 tests), Ruff, and diff check. The full Python 3.13 suite completed with 4,079 passed; its only two failures are unrelated live-PyPI pin drift in autofix-versions.env (ruff 0.15.20 vs 0.15.21; mypy 2.1.0 vs 2.2.0), left out of this scoped PR.

@github-actions

Copy link
Copy Markdown
Contributor

Provider Comparison Report

Provider Summary

Provider Model Verdict Confidence Summary
openai gpt-5.4 PASS 88% The merged PR appears to satisfy the documented acceptance criteria. It adds a dedicated typed compiler/CLI for .github/sync-manifest.yml, introduces a versioned JSON schema for the normalized plan...
anthropic claude-sonnet-4-6 PASS 72% The PR substantially addresses the core acceptance criteria: a typed compiler centralizes manifest resolution, consumers (drift checker, template validator, sync workflow) are migrated to use it, e...
📋 Full Provider Details (click to expand)

openai

  • Model: gpt-5.4
  • Verdict: PASS
  • Confidence: 88%
  • Scores:
    • Correctness: 9.0/10
    • Completeness: 9.0/10
    • Quality: 9.0/10
    • Testing: 9.0/10
    • Risks: 8.0/10
  • Summary: The merged PR appears to satisfy the documented acceptance criteria. It adds a dedicated typed compiler/CLI for .github/sync-manifest.yml, introduces a versioned JSON schema for the normalized plan, and migrates key consumers (sync workflow prepare/validation, drift checking, and template validation) to use compiled normalized output rather than duplicated resolution logic. The workflow now compiles/validates the manifest before sync fan-out, supporting the requirement that no consumer writes begin when compilation fails. The implementation also appears to enforce early failure for malformed/unsafe/duplicate/unsupported cases and keeps removals typed separately from copy entries. Tests are strong: a large new compiler test suite plus updates to drift/template validation tests cover precedence, directories, create-only/overwrite behavior, removals, duplicate targets, missing sources, and unsafe paths, which maps well to the stated tasks and acceptance criteria. Documentation and a formal schema were also updated, improving maintainability and contract clarity.
  • Concerns:
    • The review is based on the provided diff summary/visible excerpts rather than the full merged file contents, so very small implementation mismatches could remain unseen.
    • One manifest entry was removed (.github/scripts/agents-guard.js); this appears intentional but is a behavioral change worth validating separately against repository maintenance expectations.

anthropic

  • Model: claude-sonnet-4-6
  • Verdict: PASS
  • Confidence: 72%
  • Scores:
    • Correctness: 8.0/10
    • Completeness: 7.0/10
    • Quality: 8.0/10
    • Testing: 8.0/10
    • Risks: 7.0/10
  • Summary: The PR substantially addresses the core acceptance criteria: a typed compiler centralizes manifest resolution, consumers (drift checker, template validator, sync workflow) are migrated to use it, early-fail validation gates consumer writes, duplicate/missing/unsafe-path detection is implemented, a JSON schema with versioning is added, and test coverage (409 lines for compiler, updated drift/validator tests) covers the key fixture scenarios. The main gap is the template hashing migration task, which has no corresponding file changes in the diff. The unrelated JSON file change is minor noise. Overall the implementation is architecturally sound and the primary acceptance criteria are met, with the hashing task appearing deferred.
  • Concerns:
    • Template hashing task ('Make template hashing derive from the normalized plan') is listed as a task but no hash-related files appear in the diff - this task may be incomplete or deferred
    • langsmith-fleet-worker-attempt.json changes appear unrelated to the feature scope and may indicate accidental inclusion
    • The full compiler implementation (557 lines) is truncated in the diff, making it impossible to fully verify determinism guarantees (sorted keys, stable ordering) and all error-path behaviors
    • pr-00-gate.yml was cancelled on the merge commit - while post-merge CI is not disqualifying per instructions, this gate cancellation means some validation was skipped
    • Schema version is present in the JSON schema file but it's unclear from the truncated diff whether the compiler output itself embeds a schema_version field in every emitted JSON document

Agreement

  • Verdict: PASS (all providers)
  • Correctness: scores within 1 point (avg 8.5/10, range 8.0-9.0)
  • Quality: scores within 1 point (avg 8.5/10, range 8.0-9.0)
  • Testing: scores within 1 point (avg 8.5/10, range 8.0-9.0)
  • Risks: scores within 1 point (avg 7.5/10, range 7.0-8.0)

Disagreement

Dimension openai anthropic
Completeness 9.0/10 7.0/10

Unique Insights

  • openai: The review is based on the provided diff summary/visible excerpts rather than the full merged file contents, so very small implementation mismatches could remain unseen.; One manifest entry was removed (.github/scripts/agents-guard.js); this appears intentional but is a behavioral change worth validating separately against repository maintenance expectations.
  • anthropic: Template hashing task ('Make template hashing derive from the normalized plan') is listed as a task but no hash-related files appear in the diff - this task may be incomplete or deferred; langsmith-fleet-worker-attempt.json changes appear unrelated to the feature scope and may indicate accidental inclusion; The full compiler implementation (557 lines) is truncated in the diff, making it impossible to fully verify determinism guarantees (sorted keys, stable ordering) and all error-path behaviors; pr-00-gate.yml was cancelled on the merge commit - while post-merge CI is not disqualifying per instructions, this gate cancellation means some validation was skipped; Schema version is present in the JSON schema file but it's unclear from the truncated diff whether the compiler output itself embeds a schema_version field in every emitted JSON document

🔍 LangSmith Traces

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:codex Agent-created issues from Codex agent:needs-attention Agent needs human review or intervention agent:retry Add to trigger agent retry after rate limit or pause agents:keepalive Use to initiate keepalive functionality with agents autofix Opt-in automated formatting & lint remediation needs-human Requires human intervention or review verify:compare Compare multiple LLM evaluations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compile consumer sync manifest into one typed normalized plan

3 participants