Skip to content

Adopt Existing Repo Design

Jump to bottom
Priscila Saboia Moreira edited this page Jun 27, 2026 · 13 revisions

type: synthesis up: "Home_llm-wiki-memory-template" derived_from:

Adopt-Existing-Repo-Design

Design for scripts/adopt.sh, a one-shot adoption path for an existing repository to take on the llm-wiki-memory pattern non-destructively. Synthesised from issue #6 and the follow-up discussion, refined against the shared-bash-lib refactor (scripts/lib/) and the feature-flag infrastructure.

The gap

The template supports two flows today:

  • Create (instantiate.sh) — scaffold a new project; the repo root is the template
  • Update (update-from-template.sh) — pull template changes into a repo that was already derived from the template

Neither serves a repository that already exists, was never created from the template, and now wants to adopt the pattern: the repo already has its own code, history, README, .gitignore, and possibly its own CLAUDE.md. Adoption is not scaffolding, it is an additive overlay that must add the pattern's files without destroying anything the host repo already owns.

Decision: separate script, not a flag

adopt.sh is a sibling of instantiate.sh, not a mode of it. The two flows have different safety contracts:

  • instantiate.sh writes freely into a fresh template clone, the entire tree is template-owned.
  • adopt.sh writes into someone else's repo, the entire tree starts host-owned, and the script must be strictly additive plus opt-in for touches.

Bundling these into one script would force every line of code to check "am I in scaffold mode or adoption mode?", and a slip leaks into the safer flow. A separate script can be explicit and audited as one operation with one safety contract.

Authority model (inverted from the issue body)

The original issue body listed a denylist, files adopt must not touch (README.md, etc.). The discussion with @hegu-1 sharpened this:

never-touch should be computed, not enumerated. Invert the model. Do not maintain an infinite denylist. Maintain the allowlist of what adopt is allowed to create, and treat everything pre-existing as host-owned by default.

Concretely:

TOUCH        = additions ∪ (existing-files ∩ grants)
NEVER-TOUCH  = existing-files − grants − additions

Three lists, two owned by the template, one owned by the host:

List Owned by Mutable by Contents
ADD allowlist template template contributors Files adopt creates if absent; never overwrites
TOUCH grants host host owner (manual, before run) Per-file: managed-block, append-only, or merge. Adopt reads, never writes
NEVER-TOUCH (computed at runtime) (derived) existing − grants − additions

The host writing the grants file is the load-bearing trust step: the tool requesting permission cannot mint the permission.

ADD allowlist (template-owned)

What adopt.sh creates if absent. Each line is the path adopt writes; if the path already exists with the same content, the action is SKIP; with different content, REFUSE (host overrode it).

The list is the literal ADD_ALLOWLIST array in scripts/adopt.sh (27 entries as of feature/adopt-sh HEAD 8c3a579):

PATTERN ROOT (4)
  llm-wiki.md
  wiki/init-wiki.sh
  wiki/agents/discipline-gates.md
  wiki/agents/verification-gate.md

CLAUDE-CODE OVERLAY (5; fixed list, not gated by --agent yet)
  wiki/agents/claude-code/setup.sh
  wiki/agents/claude-code/templates/claude-md-snippet.md
  wiki/agents/claude-code/templates/session-start-hook.sh
  wiki/agents/claude-code/templates/posttooluse-hook.sh
  wiki/agents/claude-code/templates/memory-seed.md

CLAUDE-CODE SLASH COMMANDS + SKILLS (6; added in PR #51 after end-to-end review found CLAUDE.md template referenced them but adopt was not copying them)
  .claude/commands/wiki-experiment.md
  .claude/commands/wiki-source.md
  .claude/commands/wiki-lint.md
  .claude/skills/wiki-experiment.md
  .claude/skills/wiki-source.md
  .claude/skills/wiki-lint.md

SHARED LIBRARY (8; introduced by the bash-lib refactor)
  scripts/lib/common.sh
  scripts/lib/git.sh
  scripts/lib/identity.sh
  scripts/lib/text.sh
  scripts/lib/report.sh
  scripts/lib/claude.sh
  scripts/lib/sys.sh
  scripts/lib/install-feature.sh

UPDATE MACHINERY (2)
  scripts/update-from-template.sh
  scripts/check-template-version.sh

FEATURE-FLAG ENTRY SCRIPTS (2)
  scripts/enable-feature.sh
  scripts/disable-feature.sh

Two side-effects, NOT in the array, happen during Phase 2B:

  • wiki/<repo>.wiki/ is created by wiki/init-wiki.sh running in create-mode (Phase 2B step 1), not by an ADD copy.
  • The overlay setup script wiki/agents/claude-code/setup.sh runs in Phase 2B step 2 and may write into the host's CLAUDE.md (per the managed-block grant) and .claude/settings.json (per the merge grant). These writes are governed by the host's grants file, not by the ADD allowlist.

Apply-time honesty (PR #51 item #1): the apply loop captures mkdir -p and cp -p exit codes. Successful copies land in APPLIED_ADDS; failures (permissions, disk full, read-only mount, destination path blocked by a non-directory) land in FAILED_ADDS and surface as a dedicated - ADD FAILED (N files; cp or mkdir -p returned non-zero): block in the manifest plus a , N FAILED clause in the summary stdout line. The manifest stays honest about what is actually on disk.

Deferred surface (named explicitly because the original design sketch listed them, but they are not in the current ADD_ALLOWLIST):

  • scripts/wiki-write-protocol/ — not copied by adopt.
  • features/ and feature install via --features — adopt's dry-run prints NOT IMPLEMENTED YET: Feature install via --features as an explicit stub limitation (see issue #53 for the warning gap).

--agent=cursor errors with "not yet supported (see issue #6, deferred)"; --agent=none skips the overlay setup step entirely (managed-block and merge TOUCH grants are then recorded as skipped, never failed, never applied).

TOUCH grants (host-owned)

The host repo includes .llm-wiki-adopt-grants.yml. Without it, every pre-existing file defaults to NEVER-TOUCH and adopt only performs ADD steps.

# .llm-wiki-adopt-grants.yml
# Permissions you grant adopt.sh to touch existing files in this repo.
# Default for any pre-existing file is NEVER-TOUCH; entries here override.
grants:
  CLAUDE.md:              managed-block      # adopt inserts a fenced block; deletes are reversible
  .gitignore:             append-only        # adopt appends wiki-related rules inside a sentinel block
  .claude/settings.json:  merge              # adopt jq-merges keys (claude-code overlay needs a hook entry)

Grant types:

  • managed-block: adopt delegates to the agent overlay's setup.sh, which injects sentinel-paired blocks into the host file. For the claude-code overlay applied to CLAUDE.md, this means TWO blocks land: <!-- lw:memory-boundary --> ... <!-- /lw:memory-boundary --> and <!-- lw:wiki-maintenance --> ... <!-- /lw:wiki-maintenance -->. Deleting either block returns that section of the host file to its original authority. Mechanism: overlay's setup.sh calling lw_inject_block from the shared library, anchored relative to the host's ### Knowledge Graph header (or appended at EOF when the header is absent).
  • append-only: adopt calls lw_inject_block directly with a per-target payload anchored at end-of-file. The current append-only grant target is .gitignore with sentinel lw:wiki-rules; payload is the canonical wiki/*.wiki/ rule.
  • merge: adopt delegates to the overlay's setup.sh --hook for the jq deep-merge, currently exercised by the .claude/settings.json: merge grant which adds the SessionStart hook entry while preserving host's existing keys (host wins on conflicting keys).

Every grant is bounded: adopt never rewrites host prose, only its own fenced region. The operation is therefore bounded, idempotent, and reversible.

Absent targets are not "moot" — they are created from canonical (PR #51 items 3, 4, 5, after Chris Sweet's end-to-end review). The grant governs how to safely modify a host file, not whether to create one. When the target is absent the host has no content to preserve, so the canonical install path fires: append-only writes the sentinel-wrapped payload into a fresh file; managed-block lets init-wiki seed the file from the template then the overlay setup patches it; merge lets the overlay's setup.sh --hook create the canonical settings file from scratch. The manifest distinguishes the two paths via the status string (applied vs created from canonical). CLAUDE.md already behaved this way via init-wiki; this brings .gitignore and .claude/settings.json into the same shape.

Dry-run report (always available)

adopt.sh --dry-run (the default when --apply is absent) is the safety surface for the host owner: it reports every action by class without applying any. The report classifies each affected path so that the refusals are legible.

Verbatim shape of the real report (virgin host with grants for all three target types, all three targets absent on disk):

$ bash scripts/adopt.sh --target=/path/to/host

adopt.sh --dry-run

Target:           /path/to/host
Resolved:         <repo>   (lw_name_from_origin)
Template:         /path/to/llm-wiki-memory-template
Agent overlay:    claude-code
Features:         <none>
Grants file:      .llm-wiki-adopt-grants.yml (3 grant(s) found)

ADD  (would create N files)
  + llm-wiki.md
  + wiki/init-wiki.sh
  + ...

SKIP (already present, identical to template — N files)
  = scripts/lib/git.sh

REFUSE (host has a different version — N files)
  ✗ wiki/init-wiki.sh  (host-modified)

TOUCH (host-owned, granted — 3 files)
  ~ CLAUDE.md                        managed-block [absent; will create from canonical]
  ~ .gitignore                       append-only (sentinel lw:wiki-rules) [absent; will create from canonical]
  ~ .claude/settings.json            merge [absent; will create from canonical]

GRANT WARNINGS (entries in grants file that did not produce a TOUCH)
  ! Makefile  (unknown grant target; template has no operation for it)

NOT IMPLEMENTED YET (stub limitation, not absent from the design)
  - Feature install via --features (install_feature from scripts/lib/)

Six section headers in fixed order: ADD, SKIP, REFUSE, TOUCH, GRANT WARNINGS, NOT IMPLEMENTED YET. GRANT WARNINGS is emitted only when at least one grant could not be classified as TOUCH (entries marked ! are TOUCH_INVALID: target unknown to the template, or grant type mismatches the template's operation). Absent host targets are no longer warnings — they become TOUCH rows with the [absent; will create from canonical] marker (PR #51 redesign).

The Resolved: line shows the basename returned by lw_name_from_origin, not <owner>/<repo> (the <owner>/ prefix is stripped). The sentinel parenthetical on TOUCH rows only appears for append-only grants where adopt itself owns the sentinel name; managed-block and merge grants delegate to the overlay's setup.sh and the parenthetical is omitted (adopt does not own those sentinel names).

The why column on REFUSE rows is the contract: "host-modified" means it exists with different content from the template. A surprising REFUSE is a bug to fix in the grants file or the allowlist, never silently in the script.

Adoption manifest (.llm-wiki-adopt-log.md)

On apply, adopt writes one log file recording what it did. This closes the loop opened by hegu-1's fourth class ("adoption manifest"): the host needs an inventory of what changed on adoption day, separate from later update-from-template.sh syncs (which already log to .llm-wiki-template-log.md).

The manifest is append-only across multiple --apply runs (each run is its own dated entry). Verbatim shape of one entry as written by the current code, run against a virgin host with all three grants where every granted target is absent:

# llm-wiki adopt log

## [2026-06-25] adopt --apply (phases 1, 2A, 2B, 3)
- project: <repo>
- agent: claude-code
- signals matched: 0 of 3 (none)
- overlay(s) detected: none
- ADDed (27 files):
  - llm-wiki.md
  - wiki/init-wiki.sh
  - .claude/commands/wiki-experiment.md
  - .claude/skills/wiki-experiment.md
  - ...
- SKIPped (0, byte-equal already)
- REFUSEd (0, host-modified; left alone)
- init-wiki: applied (ran wiki/init-wiki.sh --repo-name <repo>)
- overlay setup: applied (ran wiki/agents/claude-code/setup.sh)
- TOUCH applied (3):
  - CLAUDE.md (managed-block): created from canonical and patched via wiki/agents/claude-code/setup.sh
  - .gitignore (append-only): created from canonical
  - .claude/settings.json (merge): created from canonical via wiki/agents/claude-code/setup.sh --hook

TOUCH status vocabulary, per entry: applied (host had content, the sentinel-paired block was injected into existing prose); already-present (idempotent re-apply detected the sentinel); created from canonical (host did not have the file, adopt or the overlay wrote it from the template); created from canonical and patched via ... (specific to managed-block: init-wiki seeded, overlay setup patched in one cohesive step); skipped (reason) (e.g. --agent=none, no overlay to set up); failed (...) (the delegated helper exited non-zero; see issue #54 for the stderr-diagnosis gap).

When the apply loop hits cp -p or mkdir -p failures (PR #51 item #1), an additional - ADD FAILED (N files; ...) block appears between ADDed and SKIPped, listing the paths that could not be written.

Fields recorded per entry: phases run (heading); project name resolved by lw_name_from_origin; agent overlay chosen; composite signals matched on entry to the run (count plus signal names); overlay(s) detected in the host before the run; ADD list; SKIP/REFUSE counts; init-wiki status (applied, already-present, failed, skipped); overlay setup status (same vocabulary); TOUCH applied list with per-entry status (applied, already-present, skipped, failed).

The current entry does NOT pin a template version (no SHA, no tag); a future iteration could add that field. Re-running --apply on the same host with the same grants is bounded by the advisory-abort: the composite detector will report 2-of-3 or higher and route the user to update-from-template.sh rather than re-walking the pipeline.

Decision: advisory abort when host is already adopted (added 2026-06-25)

The original idempotency claim above is partially right but understates the failure mode. adopt.sh is for first-time adoption. Re-running --apply on a host that already shows the pattern is not just redundant, it is hazardous: Phase 2B invokes the host's own copy of wiki/init-wiki.sh and wiki/agents/<overlay>/setup.sh, which may have drifted away from the template's versions (the case team-ai-Engineering surfaces). The host's drifted overlay then runs with the template's expectations, and the result is undefined behavior, not idempotent re-application.

Cleaner contract: detect the already-adopted case before any apply work, abort with an advisory, and route the user to scripts/update-from-template.sh, which knows about ALWAYS_FILES and handles template drift in a scoped way.

Detection uses three harness-agnostic signals:

  • Signal A: llm-wiki.md is byte-identical to the template's
  • Signal B: wiki/agents/discipline-gates.md is byte-identical to the template's
  • Signal C: wiki/init-wiki.sh is present

The composite threshold is 2 of 3. One signal is too weak (any of the three could be present coincidentally in a virgin host the maintainer staged manually); requiring all three is too strict (the team-ai-Engineering case has Signal B drifted but is unambiguously adopted). Two-of-three is the smallest threshold that survives the realistic mix.

--force is the escape hatch when the host owner really means a re-apply (rare; mostly for testing the apply path on a fixture that pre-stages signals). The advisory abort is the default, and the dry-run report surfaces the signal count and which signals matched so the maintainer can audit before deciding.

Empirical classifications observed during dry-run testing on 2026-06-25 and re-validated against feature/adopt-sh HEAD 8c3a579 on 2026-06-27 (all numbers from real script output, not projection):

Repo Signals Outcome Notes
crcresearch/FUNSD (fresh clone) 0/3 proceed with apply virgin, 27 ADDs
chrissweet/FHI360_Lite (fresh clone) 0/3 proceed with apply virgin, 27 ADDs; the host where Chris ran the PR #51 end-to-end review
pad-analytics-workshop 0/3 proceed with apply virgin
pad-ml-pipeline 0/3 proceed with apply virgin
tai_ner 0/3 proceed with apply virgin
CSE-60868 0/3 proceed with apply virgin
team-ai-Engineering 3/3 advisory abort, route to update derived, drifted
p28-behavioral-test 3/3 advisory abort, route to update derived
llm-wiki-branch-test 3/3 advisory abort, route to update derived, host-modified REFUSEs
markov_embeddings_and_rag 2/3 advisory abort, route to update prototype, parallel evolution
llm-wiki-memory-template (self) n/a rejected with "target is the template itself" self-target guard

All five "already adopted" classifications are correct: re-running adopt against any of them would have invoked a drifted overlay. The classifier protects them. The self-target guard catches the obvious "adopt the template into itself" mistake before the classifier runs.

End-to-end --apply was exercised against real (not synthetic) hosts in three configurations:

  • Linux scratch clone of pad-analytics-workshop (PR #51 first round): RC=0, 21 ADDs, manifest written, CLAUDE.md seeded with sentinels.
  • macOS bash 3.2.57 scratch virgin with .claude/settings.json: merge grant on absent target (PR #51 first round, the bash 3.2 hazard repro): RC=0 post-fix, GRANT WARNINGS emitted, manifest written.
  • Linux scratch clone of chrissweet/FHI360_Lite in two configurations (PR #51 review-resolution round, 2026-06-27): Test A (all three grants absent — CLAUDE.md, .gitignore, .claude/settings.json) reproducing the original Chris report: RC=0, 27 ADDs, all three TOUCH applied entries report created from canonical with the right delegation note, on-disk files materialise with the canonical content (CLAUDE.md gets four sentinels, .gitignore gets the wiki rule, settings.json gets the SessionStart hook). Test B (all three pre-authored on host): RC=0, 27 ADDs, all three TOUCH applied entries report applied, host-authored CLAUDE.md title preserved, host's *.npz and other rules preserved in .gitignore alongside the new wiki rule, host's theme: host and permissions.allow.Bash keys preserved in settings.json alongside the merged SessionStart hook.

The advisory-abort path and the apply path are both covered against real shells, not just CI sandbox. Test A and Test B together cover both halves of the absent/present dispatch matrix.

Reusable vs new

Reusable from existing infra New for adopt.sh
Identity resolution lw_name_from_origin
CLAUDE.md / .gitignore managed-block lw_inject_block (scripts/lib/text.sh)
Wiki sub-repo init init-wiki.sh create-mode
Overlay setup wiki/agents/claude-code/setup.sh
Feature install (optional via --features) install_feature (scripts/lib/install-feature.sh)
Non-overwrite copy small lw_copy_if_absent helper
jq deep-merge scripts/lib/json.sh (~30 lines)
Grants reader YAML parser in adopt.sh
Adoption log writer section in adopt.sh
Orchestrator scripts/adopt.sh entry point

Most of the machinery already exists. Adopt is principally orchestration + grants reader + adoption-log writer, plus two small helpers.

Test approach

Two layers:

Layer 1, controlled fixture, lives in the template at scripts/test/tests/integration/adopt-shape/. A small synthetic host repo with a fabricated README, CLAUDE.md, .gitignore, and .claude/settings.json exercises every class of TOUCH and REFUSE. Regression-stable, hermetic, runs in CI alongside the rest of the harness.

Layer 2, real-repo dry-run. A public repository that has not yet adopted the pattern, with real history and a real README, exercises adopt in production-like conditions. crcresearch/FUNSD (a Python + DVC project for FUNSD datasets) is a candidate: no wiki, no .claude/, no CLAUDE.md, defaults to main, has its own scripts/ (tests collision avoidance), has its own LICENSE and README (tests REFUSE). A run of adopt.sh --dry-run against FUNSD produces a real report we can inspect for surprises before any real apply.

markov_embeddings_and_rag is a third kind of test, but conceptually distinct: it is the prototype where this pattern was originally explored. The template was extracted from it and then evolved separately. The two share an idea ancestor, not a sync relationship. Running adopt.sh --dry-run against it surfaces divergence between two parallel evolutions of the same idea (REFUSE on wiki/init-wiki.sh when the template's version absorbed refinements the prototype never received, SKIP on files that happen to be byte-identical). Useful for spotting drift, not a check that the prototype "matches the template", since neither is canonical for the other.

team-ai-Engineering is a fourth distinct case: a repository that was instantiated from the template, but at a moment before later template-side changes landed (the shared bash-lib refactor in particular). Running adopt.sh --dry-run against it surfaces drift in a derived project: ADD for files the template now has but the derived doesn't (e.g. all of scripts/lib/*.sh), REFUSE for files the template rewrote since instantiation (e.g. wiki/init-wiki.sh, scripts/update-from-template.sh, scripts/check-template-version.sh all rewired to use the shared lib). This is a useful side-effect of the same machinery: adopt.sh doubles as a derived-project drift detector, complementary to update-from-template.sh's scoped file syncing. The four cases together (virgin, prototype, derived-and-drifted, hermetic fixture) cover the realistic surface adopt.sh has to be honest about.

Open questions

  • Granting CLAUDE.md ergonomics. Even with managed-block, opening someone else's CLAUDE.md and inserting a fenced section is a high-trust operation. Should the default grant template ship in the issue body, or should adopt require the host to write the grants file by hand? The "by hand" requirement aligns with hegu-1's principle that the tool cannot mint its own permission.
  • .llm-wiki-adopt-grants.yml evolution. When the template adds a new TOUCH category (say, a Makefile adopt would like to touch), existing grants files won't anticipate it. Forward compatibility could be addressed by versioned grant schemas, or by adopt simply REFUSING unknown TOUCH targets and reporting them: the latter is safer.
  • Uninstall / undo. The issue marked uninstall as out of scope. The managed-block design makes it cheap: deleting the sentinel blocks reverts the host portions, deleting the ADD-allowlist files removes the template portions. Worth a follow-up unadopt.sh.
  • Drift between adopt.sh's ADD_ALLOWLIST and update-from-template.sh's ALWAYS_FILES. Both scripts enumerate template-owned files in parallel bash arrays; they already diverge (e.g. update's list includes wiki/agents/README.md, wiki/agents/wiki-write-protocol.md, and the whole scripts/wiki-write-protocol/* tree, none of which appear in adopt's stub). Adding a new template file means remembering to edit two arrays with the right category, or one drifts past the other — the same anti-pattern PR #42 consolidated for instantiate.sh/update-from-template.sh/setup.sh. Agreed direction (deferred to after feature/adopt-sh matures, 2026-06-25): single source of truth in scripts/lib/template-manifest.sh, classifying entries as TEMPLATE_SHARED_INFRA (canonical template content; update overwrites, adopt ADDs without overwrite) and TEMPLATE_HOST_OWNED (host owns content; update ignores, adopt classifies via grants). Both scripts source the manifest. Adding a template file becomes one edit plus one category decision. The consolidation is the right move but lives in a separate follow-up issue/PR so the current adopt.sh PR stays scoped to "ship adopt's own surface".

Related work

  • The shared-bash-lib refactor introduced lw_inject_block, lw_name_from_origin, and the install-feature mechanics that adopt.sh builds on top of.
  • The agent-comms feature's enroll.sh already demonstrates the "interactive, additive, refuses-to-overwrite" pattern in a smaller setting, adopt generalises that discipline to the full pattern surface.
  • Multi-Agent-Write-Protocol documents the wiki sub-repo concurrency contract that adopt's init-wiki.sh step inherits.
  • OKF-Alignment-Ideas vector 4 covers the OKF-bundle import path: what adopt.sh --seed-from-okf=<bundle> plausibly does, why it strengthens vectors 2 and 3, and the three open follow-ups on link-syntax translation, frontmatter back-fill, and citation reconciliation.
  • Knowledge-Bundles-Framing names adopt as the "take on the pattern non-destructively" lifecycle step (one of five) and folds the seed-from-OKF mode into the bundle-lifecycle narrative.

Clone this wiki locally