Skip to content

Adopt Existing Repo Design

Jump to bottom
Priscila Saboia Moreira edited this page Jun 25, 2026 · 13 revisions

type: synthesis up: "Home_llm-wiki-memory-template" derived_from:

Adopt-Existing-Repo-Design

Design for scripts/adopt.sh, a one-shot adoption path for an existing repository to take on the llm-wiki-memory pattern non-destructively. Synthesised from issue #6 and the follow-up discussion, refined against the shared-bash-lib refactor (scripts/lib/) and the feature-flag infrastructure.

The gap

The template supports two flows today:

  • Create (instantiate.sh) — scaffold a new project; the repo root is the template
  • Update (update-from-template.sh) — pull template changes into a repo that was already derived from the template

Neither serves a repository that already exists, was never created from the template, and now wants to adopt the pattern: the repo already has its own code, history, README, .gitignore, and possibly its own CLAUDE.md. Adoption is not scaffolding, it is an additive overlay that must add the pattern's files without destroying anything the host repo already owns.

Decision: separate script, not a flag

adopt.sh is a sibling of instantiate.sh, not a mode of it. The two flows have different safety contracts:

  • instantiate.sh writes freely into a fresh template clone, the entire tree is template-owned.
  • adopt.sh writes into someone else's repo, the entire tree starts host-owned, and the script must be strictly additive plus opt-in for touches.

Bundling these into one script would force every line of code to check "am I in scaffold mode or adoption mode?", and a slip leaks into the safer flow. A separate script can be explicit and audited as one operation with one safety contract.

Authority model (inverted from the issue body)

The original issue body listed a denylist, files adopt must not touch (README.md, etc.). The discussion with @hegu-1 sharpened this:

never-touch should be computed, not enumerated. Invert the model. Do not maintain an infinite denylist. Maintain the allowlist of what adopt is allowed to create, and treat everything pre-existing as host-owned by default.

Concretely:

TOUCH        = additions ∪ (existing-files ∩ grants)
NEVER-TOUCH  = existing-files − grants − additions

Three lists, two owned by the template, one owned by the host:

List Owned by Mutable by Contents
ADD allowlist template template contributors Files adopt creates if absent; never overwrites
TOUCH grants host host owner (manual, before run) Per-file: managed-block, append-only, or merge. Adopt reads, never writes
NEVER-TOUCH (computed at runtime) (derived) existing − grants − additions

The host writing the grants file is the load-bearing trust step: the tool requesting permission cannot mint the permission.

ADD allowlist (template-owned)

What adopt.sh creates if absent. Each line is the path adopt writes; if the path already exists with the same content, the action is SKIP; with different content, REFUSE (host overrode it).

PATTERN ROOT
  llm-wiki.md
  wiki/init-wiki.sh
  wiki/agents/discipline-gates.md
  wiki/agents/verification-gate.md

SHARED LIBRARY (introduced by the bash-lib refactor)
  scripts/lib/{common,git,identity,text,report,claude,sys}.sh

UPDATE MACHINERY
  scripts/update-from-template.sh
  scripts/check-template-version.sh

WIKI WRITE PROTOCOL
  scripts/wiki-write-protocol/

FEATURE FLAGS
  scripts/enable-feature.sh
  scripts/disable-feature.sh
  scripts/lib/install-feature.sh
  features/                                    (with --features=<list>)

WIKI SUB-REPO
  wiki/<repo>.wiki/                            (delegate to init-wiki.sh create-mode)

AGENT OVERLAY (with --agent=claude-code)
  wiki/agents/claude-code/{setup.sh, templates/}
  .claude/commands/wiki-{experiment,source,lint}.md
  .claude/skills/wiki-{experiment,source,lint}/

--agent=cursor errors with "not yet supported"; deferred per the issue body. --agent=none skips the overlay block.

TOUCH grants (host-owned)

The host repo includes .llm-wiki-adopt-grants.yml. Without it, every pre-existing file defaults to NEVER-TOUCH and adopt only performs ADD steps.

# .llm-wiki-adopt-grants.yml
# Permissions you grant adopt.sh to touch existing files in this repo.
# Default for any pre-existing file is NEVER-TOUCH; entries here override.
grants:
  CLAUDE.md:              managed-block      # adopt inserts a fenced block; deletes are reversible
  .gitignore:             append-only        # adopt appends wiki-related rules inside a sentinel block
  .claude/settings.json:  merge              # adopt jq-merges keys (claude-code overlay needs a hook entry)

Grant types:

  • managed-block: insert and maintain a sentinel-paired block (e.g. <!-- lw:wiki-section --> ... <!-- /lw:wiki-section -->); deleting the block returns the host file to its original authority. Mechanism: lw_inject_block from the shared library.
  • append-only: same sentinel pattern, anchored at file end (for .gitignore and similar).
  • merge: jq deep-merge with explicit policy for conflicting keys (host wins by default).

Every grant is bounded: adopt never rewrites host prose, only its own fenced region. The operation is therefore bounded, idempotent, and reversible.

Dry-run report (always available)

adopt.sh --dry-run is the safety surface for the host owner: it reports every action by class without applying any. The report classifies each affected path so that the refusals are legible.

$ adopt.sh --agent=claude-code --dry-run

Resolved: <owner>/<repo>     (from origin via lw_name_from_origin)
Agent overlay: claude-code
Grants file: .llm-wiki-adopt-grants.yml (3 grants)

ADD  (template-owned, N files)
  + llm-wiki.md
  + wiki/init-wiki.sh
  + ...

TOUCH  (host-owned, granted)
  ~ CLAUDE.md               managed-block (lw:wiki-section)
  ~ .gitignore              append-only (lw:wiki-rules)
  ~ .claude/settings.json   merge

REFUSE  (host-owned, no grant)
  ✗ README.md               no grant
  ✗ requirements.txt        no grant

SKIP  (already present, identical)
  = scripts/lib/git.sh      identical to template

Run without --dry-run to apply ADD + TOUCH.
A manifest of what was added and touched will be written to .llm-wiki-adopt-log.md.

The why column on REFUSE rows is the contract: "no grant" means the host did not enumerate it; "host-modified" means it exists with different content. A surprising REFUSE is a bug to fix in the grants file or the allowlist, never silently in the script.

Adoption manifest (.llm-wiki-adopt-log.md)

On apply, adopt writes one log file recording what it did. This closes the loop opened by hegu-1's fourth class ("adoption manifest"): the host needs an inventory of what changed on adoption day, separate from later update-from-template.sh syncs (which already log to .llm-wiki-template-log.md).

The manifest records files added, files touched (with grant type), grants present, agent overlay chosen, template version pinned. Future adopt runs are idempotent: re-running on the same repo with the same grants either SKIPs (already present) or refuses to re-touch (already managed).

Decision: advisory abort when host is already adopted (added 2026-06-25)

The original idempotency claim above is partially right but understates the failure mode. adopt.sh is for first-time adoption. Re-running --apply on a host that already shows the pattern is not just redundant, it is hazardous: Phase 2B invokes the host's own copy of wiki/init-wiki.sh and wiki/agents/<overlay>/setup.sh, which may have drifted away from the template's versions (the case team-ai-Engineering surfaces). The host's drifted overlay then runs with the template's expectations, and the result is undefined behavior, not idempotent re-application.

Cleaner contract: detect the already-adopted case before any apply work, abort with an advisory, and route the user to scripts/update-from-template.sh, which knows about ALWAYS_FILES and handles template drift in a scoped way.

Detection uses three harness-agnostic signals:

  • Signal A: llm-wiki.md is byte-identical to the template's
  • Signal B: wiki/agents/discipline-gates.md is byte-identical to the template's
  • Signal C: wiki/init-wiki.sh is present

The composite threshold is 2 of 3. One signal is too weak (any of the three could be present coincidentally in a virgin host the maintainer staged manually); requiring all three is too strict (the team-ai-Engineering case has Signal B drifted but is unambiguously adopted). Two-of-three is the smallest threshold that survives the realistic mix.

--force is the escape hatch when the host owner really means a re-apply (rare; mostly for testing the apply path on a fixture that pre-stages signals). The advisory abort is the default, and the dry-run report surfaces the signal count and which signals matched so the maintainer can audit before deciding.

Empirical classifications observed during dry-run testing on 2026-06-25 (all numbers from real script output, not projection):

Repo Signals Outcome Notes
crcresearch/FUNSD (fresh clone) 0/3 proceed with apply virgin, 21 ADDs
pad-analytics-workshop 0/3 proceed with apply virgin
pad-ml-pipeline 0/3 proceed with apply virgin
tai_ner 0/3 proceed with apply virgin
CSE-60868 0/3 proceed with apply virgin
team-ai-Engineering 3/3 advisory abort, route to update derived, drifted
p28-behavioral-test 3/3 advisory abort, route to update derived
llm-wiki-branch-test 3/3 advisory abort, route to update derived, 7 host-modified REFUSEs
markov_embeddings_and_rag 2/3 advisory abort, route to update prototype, parallel evolution
llm-wiki-memory-template (self) n/a rejected with "target is the template itself" self-target guard

All five "already adopted" classifications are correct: re-running adopt against any of them would have invoked a drifted overlay. The classifier protects them. The self-target guard catches the obvious "adopt the template into itself" mistake before the classifier runs.

End-to-end --apply was exercised on two virgins, not just dry-run: a Linux scratch clone of pad-analytics-workshop (RC=0, 21 ADDs, manifest written, CLAUDE.md seeded with sentinels) and a macOS bash 3.2.57 scratch host with the .claude/settings.json: merge grant pattern that surfaced the bash 3.2 hazard (RC=0 post-fix, GRANT WARNINGS emitted, manifest written, .claude/ not silently created). The advisory-abort path and the apply path are both covered against real shells, not just CI sandbox.

Reusable vs new

Reusable from existing infra New for adopt.sh
Identity resolution lw_name_from_origin
CLAUDE.md / .gitignore managed-block lw_inject_block (scripts/lib/text.sh)
Wiki sub-repo init init-wiki.sh create-mode
Overlay setup wiki/agents/claude-code/setup.sh
Feature install (optional via --features) install_feature (scripts/lib/install-feature.sh)
Non-overwrite copy small lw_copy_if_absent helper
jq deep-merge scripts/lib/json.sh (~30 lines)
Grants reader YAML parser in adopt.sh
Adoption log writer section in adopt.sh
Orchestrator scripts/adopt.sh entry point

Most of the machinery already exists. Adopt is principally orchestration + grants reader + adoption-log writer, plus two small helpers.

Test approach

Two layers:

Layer 1, controlled fixture, lives in the template at scripts/test/tests/integration/adopt-shape/. A small synthetic host repo with a fabricated README, CLAUDE.md, .gitignore, and .claude/settings.json exercises every class of TOUCH and REFUSE. Regression-stable, hermetic, runs in CI alongside the rest of the harness.

Layer 2, real-repo dry-run. A public repository that has not yet adopted the pattern, with real history and a real README, exercises adopt in production-like conditions. crcresearch/FUNSD (a Python + DVC project for FUNSD datasets) is a candidate: no wiki, no .claude/, no CLAUDE.md, defaults to main, has its own scripts/ (tests collision avoidance), has its own LICENSE and README (tests REFUSE). A run of adopt.sh --dry-run against FUNSD produces a real report we can inspect for surprises before any real apply.

markov_embeddings_and_rag is a third kind of test, but conceptually distinct: it is the prototype where this pattern was originally explored. The template was extracted from it and then evolved separately. The two share an idea ancestor, not a sync relationship. Running adopt.sh --dry-run against it surfaces divergence between two parallel evolutions of the same idea (REFUSE on wiki/init-wiki.sh when the template's version absorbed refinements the prototype never received, SKIP on files that happen to be byte-identical). Useful for spotting drift, not a check that the prototype "matches the template", since neither is canonical for the other.

team-ai-Engineering is a fourth distinct case: a repository that was instantiated from the template, but at a moment before later template-side changes landed (the shared bash-lib refactor in particular). Running adopt.sh --dry-run against it surfaces drift in a derived project: ADD for files the template now has but the derived doesn't (e.g. all of scripts/lib/*.sh), REFUSE for files the template rewrote since instantiation (e.g. wiki/init-wiki.sh, scripts/update-from-template.sh, scripts/check-template-version.sh all rewired to use the shared lib). This is a useful side-effect of the same machinery: adopt.sh doubles as a derived-project drift detector, complementary to update-from-template.sh's scoped file syncing. The four cases together (virgin, prototype, derived-and-drifted, hermetic fixture) cover the realistic surface adopt.sh has to be honest about.

Open questions

  • Granting CLAUDE.md ergonomics. Even with managed-block, opening someone else's CLAUDE.md and inserting a fenced section is a high-trust operation. Should the default grant template ship in the issue body, or should adopt require the host to write the grants file by hand? The "by hand" requirement aligns with hegu-1's principle that the tool cannot mint its own permission.
  • .llm-wiki-adopt-grants.yml evolution. When the template adds a new TOUCH category (say, a Makefile adopt would like to touch), existing grants files won't anticipate it. Forward compatibility could be addressed by versioned grant schemas, or by adopt simply REFUSING unknown TOUCH targets and reporting them: the latter is safer.
  • Uninstall / undo. The issue marked uninstall as out of scope. The managed-block design makes it cheap: deleting the sentinel blocks reverts the host portions, deleting the ADD-allowlist files removes the template portions. Worth a follow-up unadopt.sh.
  • Drift between adopt.sh's ADD_ALLOWLIST and update-from-template.sh's ALWAYS_FILES. Both scripts enumerate template-owned files in parallel bash arrays; they already diverge (e.g. update's list includes wiki/agents/README.md, wiki/agents/wiki-write-protocol.md, and the whole scripts/wiki-write-protocol/* tree, none of which appear in adopt's stub). Adding a new template file means remembering to edit two arrays with the right category, or one drifts past the other — the same anti-pattern PR #42 consolidated for instantiate.sh/update-from-template.sh/setup.sh. Agreed direction (deferred to after feature/adopt-sh matures, 2026-06-25): single source of truth in scripts/lib/template-manifest.sh, classifying entries as TEMPLATE_SHARED_INFRA (canonical template content; update overwrites, adopt ADDs without overwrite) and TEMPLATE_HOST_OWNED (host owns content; update ignores, adopt classifies via grants). Both scripts source the manifest. Adding a template file becomes one edit plus one category decision. The consolidation is the right move but lives in a separate follow-up issue/PR so the current adopt.sh PR stays scoped to "ship adopt's own surface".

Related work

  • The shared-bash-lib refactor introduced lw_inject_block, lw_name_from_origin, and the install-feature mechanics that adopt.sh builds on top of.
  • The agent-comms feature's enroll.sh already demonstrates the "interactive, additive, refuses-to-overwrite" pattern in a smaller setting, adopt generalises that discipline to the full pattern surface.
  • Multi-Agent-Write-Protocol documents the wiki sub-repo concurrency contract that adopt's init-wiki.sh step inherits.
  • OKF-Alignment-Ideas vector 4 covers the OKF-bundle import path: what adopt.sh --seed-from-okf=<bundle> plausibly does, why it strengthens vectors 2 and 3, and the three open follow-ups on link-syntax translation, frontmatter back-fill, and citation reconciliation.

Clone this wiki locally