Skip to content

Repository review 2026 09 05

Jerry Holland edited this page Sep 29, 2026 · 3 revisions

Repository review — 2026-09-05

Assessed commit: 1969808c34c901520cf8142bc5ee45f7dce8a3f0 (v0.13.2).

The repository substantially implements its Git-backed laboratory goal. Its architecture and executable contracts are strong, but several implemented safety claims exceed current behavior, and sustained real-workflow value remains insufficiently evidenced for Gate B. This page preserves the audit as a dated assessment. Linked GitHub issues carry current work status and acceptance criteria; this page does not replace the tracker.

Concrete findings

Work item Finding Evidence Severity
#46 Repository-backed verifier accepts a bundle omitting all source changes Disposable-repository reproduction High
#47 Concurrent successful workspace creates lose a registry entry Deterministic two-process checkout-hook interleaving High
#48 Archive/restore deletes a clean unfinished operation journal Existing before-publish fault injection plus lifecycle commands High
#49 Source-history GC invalidates a completed hard-squash receipt Branch deletion, explicit reflog expiry and GC in a disposable fixture High
#50 Repeated cherry-pick of an ordinary Git origin fails instead of no-op Two real CLI invocations; pending empty cherry-pick remains Medium
#51 Fenced-code comments become semantic headings and alter merge boundaries Indexed committed Markdown and three-way merge reproduction Medium
#52 Per-host baseline selection identifies only the OS Source inspection; measured run does not establish hardware identity Medium

Related existing records have audit follow-ups: #19 (dogfooding), #18 (Proposed ADR versus described closure, and native fallback evidence), #24 (known provenance publication failure), #36 (remote verifier prerequisite), #31 (semantic conformance), and #42 (performance evidence). #17 remains relevant to whole-invocation telemetry.

Evaluation against the documented goal

Goal Assessment
Ordinary Git compatibility and recovery Strong foundation; #48/#49 expose recovery and retention gaps.
Causality across squash and rewrite Substantially implemented with explicit exact/advisory categories; #49/#50 weaken durability and adoption.
Forecast and reviewed application Strong pinning, stale checks, conflict signatures, final-tree validation, and abort tests.
Parallel AI workspaces Individual workflows work; #47 exposes unprotected shared-registry mutation.
Canonical Markdown and sparse metadata Useful incremental indexing/migration model; #51 needs corrected boundaries.
Reliable machine contracts Strong schema/compatibility/error/parity discipline; #46 is a semantic validation hole.
Evidence-led native investment Gates are sensible; this audit supplies no proof that their entry conditions have cleared.

Strengths to preserve: separation of exact proof from advisory equivalence; explicit resolution approval; pinned object identities and final-tree equality; versioned schema and compatibility checks; deterministic competing-notes-writer coverage; optimized Git equality tests and visible fallback reasons.

The main testing blind spot is feature composition. Lifecycle and interrupted publication are tested individually, but their intersection loses a journal. Competing note writers are tested, but competing registry writers are not. Missing objects are validated, but normal eventual collection of dependencies of a successfully published receipt is not prevented.

Validation performed

Local host: Linux, Node v26.8.1, Git 2.55.0.

Check Outcome
npm test 152 passed, 0 failed, 1 expected skip
VLAB_GIT_SESSION=1 npm test 152 passed, 0 failed, 1 expected skip
VLAB_GIT_SESSION=0 npm test 152 passed, 0 failed, 1 expected skip
VLAB_FORECAST_ENGINE=worktree npm test 152 passed, 0 failed, 1 expected skip
VLAB_FORECAST_ENGINE=merge-tree npm test 152 passed, 0 failed, 1 expected skip
VLAB_ENGINE=native npm test 152 passed, 0 failed, 1 expected skip; fallback qualification only
Current-commit CI Windows/Linux matrix and static checks passed
npm run test:benchmark Passed selected Linux baseline; host-identification caveat in #52
All six maintained demos Passed in a dedicated disposable temporary root
Syntax 56 JS/MJS files passed node --check
Documentation links, agent mirrors, whitespace Passed
Packed artifact Installed outside source checkout; version/help smoke passed

The local full-suite runs were sequential by mode. Demo fixtures, package-install fixtures, and destructive reproductions were isolated and cleaned. No surviving vlab session workers were observed after completion. The checkout remained unchanged, including its pre-existing untracked package-lock.json. This is bounded audit evidence, not a complete production-readiness/security assessment or a replacement for real agent sessions.

Product-evidence limits and order

The 93 commits reachable from this checkout have zero Change-Id and zero Landing-Mode trailers (#19). That does not establish absence of use elsewhere; it means this history does not demonstrate sustained use of the central workflow. Native mode currently falls back for every read (#18). Synthetic benchmark success and green tests do not establish the user-value conditions or justify a native fact store/service.

Recommended sequence: correct #46–#49 and the known partial-publication failure #24, close the ordinary-Git/Markdown boundaries #50/#51, then collect actual branch-and-land, forecast/recovery, resolution-reuse, and corpus evidence on identified hosts. #52 improves the measurement control. Ratify or reconcile ADR-0024's decision state through #18. These are recommendations against the existing gates, not new native authorization.

Gate B remains an evidence gate. Populate the nine-condition table with actual schemas, hosts, runs, outcomes, and limitations before treating it as satisfied.

Detailed findings at the assessed commit

The following reproductions and scopes preserve the original review findings. Refer to each linked issue for later corrections and implementation progress.

Reject proof bundles that omit or misidentify source changes

Work item: https://github.com/jwh3times/causet/issues/46

Problem and impact

High severity. Repository-backed verification accepts a bundle that omits all source work. This undermines FR-PLAN-08 and the explainable-complete-plan contract; it is a validation defect independent of the deferred cryptographic trust model.

Reproduction and observed result

  1. Create an ordinary repository with a base commit on main.
  2. Branch feature, add one file, and commit it with vlab commit.
  3. Switch to main and generate vlab proof-bundle feature --json into a file outside the worktree. Its counts report one new change.
  4. Set changes to [] and counts to {"covered":0,"candidate-equivalent":0,"new":0}. Leave source/target heads, lineage, and evidence unchanged. Recompute integrity.bundleHash using the exported bundleHash function.
  5. Run vlab verify-proof <file> --json from the repository, without --offline.

Observed: exit 0, classification.changes: 0, classification.agrees: true, repository.checked: true, repository.matches: true, and ok: true.

Cause

The repository comparison verifies only coverage evidence. Classification iterates only the supplied changes. Neither binds a complete source inventory, commit-to-Change-Id correspondence, summary counts, or bases to the repository.

Source: repository comparison, classification loop.

Acceptance criteria

  • Repository-backed verification rejects omitted, duplicate, injected, or misidentified source changes, including a correctly rehashed document.
  • Counts and physical/effective bases are verified for consistency and against the applicable repository facts.
  • Offline verification states its actual evidence/completeness limitations; it never claims repository-backed completeness.
  • Add adversarial integration fixtures alongside the existing doctored-status and fabricated-evidence tests.
  • Preserve independent verification rather than merely comparing two invocations of the same planner.
  • Update contracts/conformance and an ADR if persisted proof evidence must change.

Gate and order

Nothing — ready to work. Correct the existing verifier before treating #36's remote verifier as established. No native engine, signature service, or Gate B work is required.

Serialize workspace registry mutations to prevent lost concurrent registrations

Work item: https://github.com/jwh3times/causet/issues/47

Problem and impact

High severity. Two successful workspace create invocations can leave two Git worktrees but only one registered workspace. The missing entry loses its stable workspace ID, owner/focus, and lifecycle context. This affects the central parallel-agent workflow (FR-WS-01/04, GP-06).

Deterministic reproduction

  1. Initialize a disposable repository with a committed main.
  2. Install a disposable post-checkout hook that writes a per-worker arrival file, then waits for a common release file. Give each child a different worker identifier through its environment.
  3. Start vlab workspace create a --path <a> --json and vlab workspace create b --path <b> --json concurrently.
  4. Wait until both hooks have arrived; both invocations have now read the old registry and are inside worktree creation.
  5. Release both hooks and wait for completion.

Observed: both exit 0; git worktree list --porcelain lists a and b; <common-git-dir>/vcs-lab/workspaces.json contains only b in the reproduced interleaving.

Cause

Create reads the full registry before materialization and writes its stale snapshot afterwards. Atomic JSON rename prevents torn writes but does not serialize read-modify-write. Other lifecycle mutations share the same whole-registry write pattern.

Source: createWorkspace, atomic JSON writer.

Acceptance criteria

  • Serialize the full registry read-modify-write sequence across all registry writers, with defined behavior for failed materialization.
  • Two concurrent creates preserve both entries and both ordinary worktrees.
  • Concurrent lifecycle operations on different entries cannot overwrite each other's updates.
  • Add deterministic interleaving coverage, modeled on the existing competing-notes-publisher test.
  • Define lock refusal and stale-holder recovery without deleting another writer's state.

Gate

Nothing — ready to work. A local registry fix does not require a daemon, native store, or Gate B.

Prevent workspace archive from deleting an unfinished operation journal

Work item: https://github.com/jwh3times/causet/issues/48

Problem and impact

High severity. A clean working tree can still contain an unfinished vlab operation. Archive removes its Git administrative directory and private journal, losing the information needed for vlab recovery/publication. This violates GP-10, NFR-DUR-02, and the reversible lifecycle intent of ADR-0012.

Reproduction and observed result

  1. Create a disposable repository with a committed main and a feature branch containing one new vlab commit.
  2. From main, create workspace worker.
  3. In that workspace run reconciliation with VLAB_TEST_FAULT=reconcile:before-publish.
  4. The command exits 70. The worktree is clean, the applied commit exists, and vlab reconcile --status --json reports active: true, with its starting commit and application journal.
  5. From another worktree, run vlab workspace archive worker --json, then vlab workspace restore worker --json.

Observed: archive and restore both exit 0. After restore, reconciliation status is {"active":false,"state":"idle"}. The retained branch is present, but the journal and vlab abort context have disappeared.

Source: archive preflight and removal, private reconciliation state.

Acceptance criteria

  • Archive checks reconciliation/rebase journals before removal, even with a clean index/worktree.
  • Refuse archival while an unfinished operation exists, unless a separately specified preservation/recovery design is implemented.
  • A refused archive preserves branch, worktree, journal bytes, and recovery behavior.
  • Add cross-feature tests for clean interrupted publication and a clean forecast mismatch, for both operation families where applicable.
  • Audit move, repair, and prune for compatible treatment of private state; distinguish demonstrated archive loss from any additional findings.

Gate

Nothing — ready to work. The minimal safe refusal implements existing recovery guarantees.

Retain Git objects required to validate completed hard-squash receipts

Work item: https://github.com/jwh3times/causet/issues/49

Problem and impact

High severity. A hard-squash receipt remains in notes while its required source commits can be collected. Validation then quarantines the completed receipt. Retaining causality after topology loss is a central product goal (FR-LAND-03/04, GP-01, NFR-DUR-01).

Reproduction and observed result

  1. Initialize a disposable repository with a base on main.
  2. Create feature, add a file, and commit with vlab commit.
  3. Switch to main, run vlab hard-squash feature, and validate metadata: one accepted portable receipt, zero errors.
  4. Delete feature, expire all reflogs with git reflog expire --expire=now --all, and run git gc --prune=now.
  5. Run vlab metadata validate --json.

Observed: the source commit no longer exists. Validation reports two missing-referenced-object errors (absorbedCommits and sourceHead), zero accepted portable records, and one quarantined record.

The explicit expiration/GC accelerates eventual cleanup of unreferenced history. This experiment must only run in a disposable repository.

Cause and scope

The note's JSON references do not make its referenced commits reachable to Git GC. Publication attaches the receipt to the landing but creates no source-retention edge. Export constructs retention parents for its envelope; that does not establish a durable local publication-time retention contract.

Source: landing publication, required-object acceptance, export retention.

Acceptance criteria

  • Specify and implement a durable retention strategy for the object closure required by accepted completed facts.
  • Define retention lifecycle and how it interacts with aborted/orphaned operations; retention must not grant target coverage to an otherwise unreachable receipt.
  • Successful hard-squash receipts remain valid and exportable after source branch deletion, reflog expiration, and GC.
  • Audit analogous rewrite/application/provenance dependencies and add targeted GC tests for affected families.
  • Preserve stock Git inspectability, exact target-reachability coverage, and safe metadata transport.

Gate

Contract needed: accept the retention/lifecycle decision in an ADR before changing persistence. Design and regression reproduction can begin now. This is a Git-backed correctness requirement, not approval for a Gate B native store.

Honor receipt-backed duplicate suppression for cherry-picks of ordinary Git commits

Work item: https://github.com/jwh3times/causet/issues/50

Problem and impact

Medium severity. Repeating a vlab cherry-pick of a Git-authored commit without a Change-Id does not return the expected covered-change no-op. It leaves a pending empty cherry-pick. This is an adoption/coverage inconsistency involving FR-ID-04/05.

Reproduction and observed result

  1. Create a disposable repository with base main.
  2. Branch feature, add a file, and use stock git commit without a Change-Id. Save the source OID.
  3. Switch to main and run vlab cherry-pick <source-oid> --json successfully.
  4. Run the same command again without --repeat.

Observed: exit 1, vcs-lab.error/v1 with code: conflict-blocked, Git reports an empty cherry-pick, and CHERRY_PICK_HEAD is present.

Cause

The origin obtains the documented git:<oid> fallback. coveredChangeIds scans only Change-Id/Absorbs trailers and ignores the exact application record created by the first invocation. The standalone cherry-pick path therefore differs from the richer causal coverage machinery.

Source: coverage scan and no-op guard.

Acceptance criteria

  • A second pick of an exactly covered ordinary-Git origin returns a no-op and leaves HEAD/index/worktree/Git operation state unchanged.
  • Cover both direct ancestry and exact application evidence where applicable.
  • Share or consistently implement the accepted coverage rules; patch similarity remains advisory.
  • Preserve explicit --repeat behavior and intentional fork semantics.
  • Add real-CLI fixtures using commits created without vlab or a Change-Id trailer.

Gate

Nothing — ready to work.

Make semantic Markdown boundaries aware of fenced code blocks

Work item: https://github.com/jwh3times/causet/issues/51

Problem and impact

Medium severity. The semantic parser treats a # comment inside fenced code as a document heading. That creates false merge units and can change code-example bytes during semantic rendering, weakening FR-SPEC-01/02/08 and the conservative section-level merge contract.

Reproduction

Index and commit this document and its manifest:

# Real section

Before.

```python
# Example comment
print(1)
```

Create two branches from this base. On the target, replace Before. with Target edit.; on the source, replace print(1) with print(2). Reindex and commit each branch, then call planSpecMerge with the base/target/source revisions.

Observed:

  • Indexed sections: Real section and Example comment, although the latter is inside a fence.
  • Merge status: clean; counts: one ours-edit and one theirs-edit.
  • Rendered output combines the edits and inserts a blank line between the opening Python fence and its comment.

Both edits belong to one actual Markdown heading section. The parser's regex has no fence state.

Source: parseBlocks, rendering.

Acceptance criteria

  • Fenced code content does not create heading or requirement entities.
  • Cover backtick/tilde fences, valid fence lengths/indentation, and unclosed fences under a specified supported syntax.
  • Same-section divergent edits remain blocked according to the conservative contract.
  • Rendering preserves code-block bytes except explicitly documented normalization.
  • Specify versioning/migration for entity boundaries affected by the correction before changing the frozen parser behavior.
  • Feed these fixtures into the semantic-merge conformance work in #31; no second-format adapter is needed for this fix.

Gate

Contract needed: agree corrected parser boundaries and version/migration treatment before implementation. Contract and reproduction work can begin now.

Key benchmark baselines by an identified host instead of operating system alone

Work item: https://github.com/jwh3times/causet/issues/52

For implemented behavior and current commands, see Benchmark host baselines. The finding below describes the assessed commit.

Problem and impact

Medium severity; confirmed by source inspection. Documentation describes a per-host benchmark baseline, but selection uses const hostKey = process.platform. Every Linux machine compares with hosts.linux, and every Windows machine with hosts.win32, regardless of hardware or identity. An unrecognized machine with a recognized OS cannot take the documented no-host-baseline skip path.

This can make release regression decisions reflect host differences instead of code changes, and weakens the representative-host evidence used by NFR-PERF-06 and ADR-0017.

Source: host key, baseline selection.

Evidence and limits

The 2026-09-05 Linux check passed the committed linux entry recorded on 2026-09-01 (Node v22.23.2/Git 2.55.0). The audit used Node v26.8.1/Git 2.55.0. The script did not establish that this was the same physical baseline host. This issue does not claim those machines are different; it identifies the missing control.

Acceptance criteria

  • Select a baseline using an explicit stable host identity or a documented host-selection mechanism, with hardware/environment provenance useful for review.
  • An unknown host on a known OS is clearly skipped/refused for latency regression purposes rather than silently borrowing another machine's entry.
  • Keep deterministic semantic/process-count checks distinguishable from host-specific latency evidence.
  • Migrate existing entries deliberately, without silently re-recording or relaxing thresholds.
  • Add comparator/selection tests using two host identities with the same OS.

Gate

Nothing — ready to work. Coordinate the resulting evidence contract with #42; no new native subsystem is required.

Original publication

Audit assessment and validation record on issue 41.

Clone this wiki locally