Skip to content

scaffold: CI test for codev/ ↔ codev-skeleton/ parity (catch drift when only one tree is edited) #1016

@amrmelsayed

Description

@amrmelsayed

Surfaced during #1011, where every protocol/role change had to be hand-applied to both the self-hosted codev/ tree and the shipped codev-skeleton/ tree. There's no automated guard against editing one and forgetting the other.

Problem

codev is self-hosted, so framework files live in two trees:

  • codev-skeleton/ — the shipped template (packaged via copy-skeleton, installs into user projects; resolver tier 4).
  • codev/ — this repo's own instance (resolver tier 2, used by this repo's builders/agents).

Changes to protocols / roles / resources must be applied to both by hand. Today that's manual/by-convention, verified ad-hoc with grep/diff during review. Nothing in CI catches "edited codev-skeleton/protocols/X but forgot codev/protocols/X" (or vice versa), so the trees can silently drift.

Existing partial guard: baked-decisions.test.ts already asserts the "Baked Decisions" paragraph is byte-identical across both trees — a precedent for section-level parity, but scoped to that one paragraph.

Proposal

A CI unit test asserting parity between codev/ and codev-skeleton/ for the files that should match, with an explicit, documented exceptions list for the known intentional differences. Failing CI when a should-match file diverges turns the current by-convention discipline into an enforced one.

Design decisions for the implementer

  • Granularity: byte-parity over an allowlist of should-be-identical files, vs. default-parity-with-exceptions, vs. section-level parity (the baked-decisions precedent). Byte-parity-with-exceptions is probably simplest, but the trees diverge enough that the exceptions list needs care.
  • Authority/severity: which tree is canonical when they differ? Fail (not warn) in CI for the should-match set.
  • Scope: protocols only, or also roles/ and resources/?

Known intentional differences to encode as exceptions

(observed during #1011 — not exhaustive)

  • Skeleton builder-prompts carry Multi-PR Workflow / Verify Phase sections that the codev/ builder-prompts don't.
  • codev/protocols/spir/protocol.md has porch approve gate examples the skeleton lacks.
  • codev/protocols/maintain/templates/ has audit-report.md + lessons-learned.md not present in the skeleton (only maintenance-run.md is in both).
  • codev/protocols/release/ exists but is not shipped in the skeleton.

Out of scope

  • Changing the two-tree model itself, or introducing auto-sync / single-source codegen between the trees — a bigger architectural question; this issue only adds a drift guard.

Suggested protocol

PIR or SPIR — it needs a brief design decision (the exceptions model + which tree is canonical) before implementation; the coding itself is small.

Related: #1011, #1013.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/scaffoldArea: Install path — codev init/adopt/update/doctor, codev-skeleton, four-tier resolver

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions