4.4.1 - setup installs with a clean environment

4.4.1 — setup installs with a clean environment

Running setup through npx — the documented way — put the parent npm's resolved configuration
into npm_config_* variables, and the nested npm install inherited them. With an npm that
supports an allowScripts switch the child aborted with:

EALLOWSCRIPTS: --allow-scripts is not allowed in project-scoped installs

so the install was skipped (only a warning said so) and you were told Next: npm install && npm run snapshot — a first-time user hitting the one thing setup exists to do.

The nested run now gets the caller's environment minus npm_config_* and npm's lifecycle
variables, so it reads the project's own .npmrc and flags.

A recording package-manager shim on PATH, with npm_config_allow_scripts=1 in the environment,
pins it: the shim is invoked and none of npm's config reaches it. The test fails without the fix.

Verified: npm run verify green — lint 105 files, 506 tests / 502 pass / 0 fail / 4 host
skips, tarball 23 files; the same suite on Node 18.20.4 and 20.19.4 (501 pass, 0 fail); and the
fixed binary installs and generates with a deliberately polluted environment:
✅ Installed dependencies (npm install, 1.1s) → Next: npm run snapshot.

4.4.0 - setup: one command for the whole first run

4.4.0 — one command for the whole first run

npx @ankurhalder/codebase-snapshot setup

That is now the whole first run, with defaults and no prompts: detect the project, write the
commented config, add the npm scripts and the devDependency, install them, and generate the
snapshot.

🚀 Setup for: /path/to/repo
✅ Created snapshot.config.ts (preset: nextjs)
   Added npm scripts: snapshot, snapshot:validate
   Added devDependency: @ankurhalder/codebase-snapshot ^4.4.0
   Added to .gitignore: snapshot/, snapshot.tmp/
⏳ Installing dependencies…
✅ Installed dependencies (npm install, 2.1s)
✅ Snapshot generated in 0.16s:
   - Tracked files: 128
   …
   Next:  npm run snapshot

What it does, precisely

Step	Detail
Config	written only when the repository has none — an existing file is kept byte for byte and the output says ♻ Keeping snapshot.config.ts
Scripts	snapshot and snapshot:validate, only when those keys are absent
Dependency	added to devDependencies unless it is already declared, with your own pin left alone
Ignore	snapshot/ and snapshot.tmp/, appended once
Install	with the package manager your lockfile names — bun.lockb, pnpm-lock.yaml, yarn.lock, otherwise npm
Generate	through the real generate command, so every guard, warning and JSON field is identical to a plain run

• --no-install skips the install: CI, an offline machine, or a lockfile you would rather no
  package manager touched. A failed install is a warning, not a dead end — the scripts resolve
  the package through npx, so the snapshot is generated either way.
• --dry-run prints the plan and the config it would write, and writes nothing.
• --format json emits one document: { "setup": { configPath, configReused, preset, scripts, dependency, gitignoreEntries, install, notes }, "manifest": { … } }.
• The ownership guard is untouched. An outDir holding files this tool did not write is still
  refused (exit 1, naming the entries) — after the config is written, so nothing is lost.
• init is unchanged. It still refuses when a config exists and needs --force; setup is the
  merge-friendly path.

Also in this release: a dead end removed

generate refused with "it holds no .codebase-snapshot marker (found parts, reports) — remove the
directory yourself" for a directory that held no files at all — just parts/ and reports/
from an interrupted run. Replacing a tree with no files destroys nothing, so it is treated as empty
now. The walk is iterative, a symlink counts as content, and a directory holding even one file is
still refused without a marker.

Verified

• npm run verify green: lint 105 files, 505 tests / 501 pass / 0 fail / 4 host-dependent skips, tarball 23 files.
• The same suite on Node 18.20.4 and 20.19.4: 500 pass, 0 fail.
• The full flow run for real against a fixture, including npm install (2.1 s) and a generated snapshot.
• This release runs the 18 / 20 / 22 verify matrix before the publish job is allowed to start.

Full details: CHANGELOG.md

4.3.0 - the audit release

4.3.0 — the audit release

A full audit of this package — every module read line by line, eight real repositories run end to
end, and adversarial probes of the filesystem, the config loader, the CLI contract and the published
tarball — produced 42 verified defects. This release fixes them, plus the six that needed a decision
first. The three that mattered most were in the two promises this package leads with.

Never leak secrets

• A quoted credential containing an escaped quote kept its tail in clear — {"password": "p\"ss"} was masked to "[REDACTED]"ss", most of the value readable and the line broken. The whole value is masked now.
• An unquoted value stopped at %, so a percent-encoded token shipped its tail (?access_token=abc%2Edef lost abc). %, : and ~ are token characters.
• A .br file was emitted as 341 bytes of U+FFFD from a 186-byte source. .br is in the binary list, and a file that is at least a third undecodable is binary too.
• Base64-wrapped PEM keys (LS0tLS1CRUdJTi…) are now detected by prefix, without decoding.

Machine-readable output

• Every usage error wrote 0 bytes to stdout in JSON mode while the docs promised one document per invocation. Usage errors now emit the error document too.
• An async hook that logged after an await appended text after the JSON payload and the command still exited 0. The redirect is scoped with AsyncLocalStorage; the payload is written once to the real stdout, even when the hook rejects.

Honest artifacts

• audit.md said "no files exceeded the large file budget" while a third of a document was missing, because the cut came from maxPartBytes. Section 3 reports both causes with a Cut by column, and generate warns when a budget cut actually happened.
• A part-budget cut carried no omission marker at all; every budget cut now names the budget.
• The preview tail window lost lines on newline-terminated files — a 10-line file at 2/2 emitted no tail. Fixed, with an honest omission count.
• A filename could inject a live Markdown link into reports/navigation.md (b](b).ts → [src/b](b)). Labels are escaped; anchors unchanged.
• full-codebase.json explains the one field it cannot carry: artifacts.fullJson is null inside it, with a selfReference pointing at manifest.json for the real digest.

Robustness

• A ~6,000-deep tree overflowed the stack after 76 s and wrote nothing; the walk is iterative and capped at 512 levels with a marker and a warning.
• One >512 MiB text file aborted the run with a raw V8 message and no skip record; it is a per-file read-error skip now.
• --discovery walk ignored .gitignore escapes (\#, \!, escaped trailing space) and included files git ignores. The parser follows gitignore(5); the matcher is escape-aware.
• init wrote branch: "main" on every repository, producing a permanent validate warning on any master/vNext default. It writes the branch git reports.
• validate, report and diff disagreed about what a manifest is; one shared guard requires generatedAt, schemaVersion, a 64-hex fingerprint, counts, totals and the files/skipped/parts arrays.
• diff ignored schemaVersion, a deleted parts array and changed part digests.
• A config carrying an unknown key, or exporting null, silently ran on the built-in defaults. Both are errors now.
• requirements.txt is classified data instead of doc.

Verified on every supported Node version

engines has said >=18; until this release nothing had ever executed on 18 or 20. The publish workflow now runs the full gate on 18, 20 and 22 for every release, and the first honest run found a real defect: on Node 18 a Windows Deny-Read ACE makes existsSync answer false, so validate called an unreadable file "missing from disk". Fixed, and verified on 18.20.4 and 20.19.4.

Full details: CHANGELOG.md

4.1.1 - a clean install, and a config that actually configures

4.1.1 — a clean install, and a config that actually configures

This one is mostly about what the tool leaves behind in your repository.

init now leaves exactly what it should

• It records the devDependency. init wrote npx @ankurhalder/codebase-snapshot …
  scripts and no dependency at all, so npm install installed nothing: every
  npm run snapshot resolved @latest out of the registry instead of a pinned copy, and the
  npm install && npm run snapshot it printed as the next step could not have installed it.
  A range you already declared — including a deliberate pin — is never overwritten.
• The config is always complete. --minimal wrote only the settings that differed from the
  defaults, so a Next.js project received export default { preset: "nextjs" } — 179 bytes
  listing none of the twenty-six settings — in the one file you open to change them. The flag is
  gone; describe no longer advertises it either.
• The footprint is snapshot.config.ts, two npm scripts, the devDependency and .gitignore.
  Nothing else: no markdown, no agent files.
• Your package.json keeps its formatting — its own indentation and line endings, so the
  diff is the change rather than the whole file.

skill --install no longer writes AGENTS.md

AGENTS.md is your project's agent contract — build commands, conventions, house rules — not a
registry for third-party skills. The install used to create it when missing (a file whose name
claims the repository documents itself, holding only this tool's text) and merge a marked
section into it when present, editing a tracked file on your behalf.

It now writes only the two registries keyed by skill name —
.claude/skills/codebase-snapshot/SKILL.md and .cursor/rules/codebase-snapshot.mdc — and
appends exactly what it creates to .gitignore, so a one-line command no longer turns into a
decision about which untracked files to commit. A file that already existed is never ignored,
and a section left behind by 4.1.0 is reported rather than rewritten. --target agents was
removed; to put the skill in your own AGENTS.md:

npx @ankurhalder/codebase-snapshot skill >> AGENTS.md

Also since 4.1.0

• Byte and line accounting. includedBytes counted the newline the closing fence swallows
  (6,290 of 6,344 files in a real repository were off by one), a CRLF checkout was over-counted
  by a byte per line, and a preview reported one line too many. Every ### FILE: block in every
  part is now compared against the manifest by an invariant test.
• A large file is no longer parsed for a summary the snapshot cannot carry: on a 1.9 MB
  source file, 1.8 s / 283 MB peak RSS became 0.5 s / 105 MB, and the omission is one grouped
  warning instead of a quiet discard.
• --format json failures write one parseable {"error":{…}} document instead of an empty
  stdout.
• Redaction stops at the path boundary: https://example.com/v1/<key> no longer becomes
  https://example.[REDACTED], and JWTs (including %2E separators) are caught.
• explain answers a file inside an ignored directory with the ancestor's record.
• init survives a .gitignore that is a directory instead of aborting after writing the
  config, and a package.json#snapshot array is a reported error.
• Discovery can be forced with --discovery walk|git|auto.

Full details: CHANGELOG.md

0.9.1 - documentation that cannot drift

The documentation now derives its version from package.json instead of restating it.

Fixed in this pass:

• SECURITY.md advertised "4.x supported" on a project whose latest release is 0.9.0. Supported
  versions are now 0.9.x, with a sentence explaining why 0.9.0 is newer than 4.4.1 rather than older.
• Both setup examples printed 4.4.0 and 4.1.1 as the dependency the command adds. They now
  track the real version automatically.
• The 0.9.0 changelog section linked the docs at a domain with no DNS record.

The mechanism: docs carry marked spans (<!--@majorMinor-->0.9<!--/@majorMinor-->) that
scripts/sync-docs.mjs rewrites from package.json; code-block examples are matched by shape,
because inside a fence an HTML comment is literal text. npm run verify fails when anything is out
of step, and npm run site and prepack both sync first — so the docs site and the published README
cannot contradict the package they describe. This release is the proof: the version was bumped, and
the examples followed on their own.

Changelog history is deliberately excluded — it records what happened at each version.

npx codebase-snapshot@latest setup

Verified: lint clean, 506 tests / 502 pass / 0 fail on Node 18, 20 and 24, tarball 28 files.

0.9.0 - first release under codebase-snapshot

First release of the unprefixed codebase-snapshot name, at 0.9.0.

npx codebase-snapshot setup

Why 0.x and not 4.5.0

codebase-snapshot is a new identity, so it starts its own line rather than inheriting the scoped
package's 4.x numbers. 0.x means what 0.x means: the CLI surface may still change in a minor release.
1.0.0 is reserved for when it is settled. Until then, pin an exact version if you need a fixed
surface — npx codebase-snapshot --version prints the current one.

The previous name, @ankurhalder/codebase-snapshot, is deprecated and points here. It still resolves.

What is in this release

• Phrase-first npm metadata: 20 keywords and a description that names the clients people search for.
• 20 GitHub topics led by ownable ones (context-engine, codebase, repo).
• README: relative links made absolute — every docs link was a 404 on npmjs.com — and build
  provenance stated.
• Documentation site generated from this repository's docs/, with sitemap, its own robots.txt,
  llms.txt and a Markdown twin per page. The wiki is closed; it was a third uncanonical copy.
• New guides: feeding a codebase to ChatGPT/Claude/Cursor, budgeting context, and honest comparisons
  with repomix, gitingest and code2prompt that each say where the alternative is better.
• npm run seo-rank measures search position and runs weekly.

Verified: lint clean, 506 tests / 502 pass / 0 fail on Node 18, 20 and 24, tarball check clean.

4.1.0 — encoding honesty, ownership evidence, discovery modes and a diff that reports payload changes

A single release carrying the whole night's work: honesty about encodings, ownership and state, no silent data loss, bounded memory, two hard crashes fixed, a new --discovery mode, and a diff that reports when the snapshot handed to a model changed even though the sources did not.

Full detail below. Upgrading is additive — one new skip reason, two new optional manifest fields and one new config setting; nothing removed, nothing required.

A UTF-16 source file was invisible, and a Latin-1 one was quietly mangled

Two encoding decisions were silent, and one was wrong in both directions. generate decided
"binary" from a NUL in the first 8 MB and then decoded with buffer.toString("utf8"), so a UTF-16
source file — which is half NUL bytes — was skipped as binary and vanished from the snapshot,
while a Latin-1 file was emitted with U+FFFD in place of every non-ASCII byte, and validate then
reported the snapshot as current because the hash covers the raw bytes.

Decoding is now explicit. A BOM, or the NUL padding of a BOM-less UTF-16 file, is recognised and
the file is transcoded; a file that is not valid UTF-8 is kept, decoded lossily, and labelled
encoding: "utf-8-lossy" with the count of undecodable bytes in warnings; a NUL anywhere in the
file — including past the 8 MB head scan, which used to be the end of the check — marks it binary
and skipped[] records the offset. files[].encoding is present only when a file was not plain
UTF-8, so its absence keeps meaning "clean UTF-8". explain answers the same question the same
way. Verified: 12 new tests covering both endiannesses, BOM-less detection, five classes of
invalid sequence, a NUL at byte 8388608, and hash stability across two runs.

A file the tool did not write could be deleted without a word

Ownership checking had a hole with a nice name. .gitkeep and .gitignore were treated as
placeholders and subtracted from the set of entries belonging to somebody else, so a snapshot
directory this generator owned and that a user then added a .gitignore to was replaced — atomically,
exit 0, manifest.warnings empty — and the file was gone. A directory holding nothing but
.gitkeep, or nothing but a .gitignore (*, !.gitignore), was replaced without any ownership
evidence at all, because the placeholder shortcut fired before the marker was ever read.

Both are files a user creates; the generator never writes either into an output directory. They are
foreign entries now, so the run refuses and names them, exactly as it already did for notes.md.
.DS_Store remains the only tolerated entry, because Finder writes it and it carries no intent.
Verified: 6 new tests — owned directory plus .gitignore, bare .gitkeep, bare .gitignore, the
refusal naming the entry, .DS_Store still discarded, and a genuine snapshot still replaced.

A directory named snapshot disappeared, and the manifest said nothing

The output directory was excluded from the scan by name, at every depth: snapshot,
snapshot.tmp and snapshot.backup-* were dropped before the branch that records a skipped[]
entry, so a directory the user committed under one of those names was missing from the snapshot
with no record anywhere — the exact omission skipped[] exists to prevent, and it contradicted
the documented promise that every discovered path is accounted for. generate and explain also
disagreed: one dropped the path, the other called it not-discovered.

Exclusion is now evidence-based. A directory is this tool's output when it carries the
.codebase-snapshot marker, and only the exact outDir/tmpDir paths a run is about to write are
excluded by name — so a first run still does not record its own output (two identical runs keep
producing identical manifests) while a user's directory with the same name is scanned like any
other. snapshot and snapshot.tmp also left the built-in ignore list, where they had been
second-layer protection for the same mistake. Verified: 5 new tests, including the same fixture
through the git and the walking discovery paths, a full generate that now lists
snapshot/keep.md, and the determinism guard that the second run does not record the first.

A hook could kill a run that had already succeeded

hooks.afterGenerate ran after the snapshot was swapped into place. A hook that returned a
rejected promise was never awaited and never caught, so Node turned it into an unhandled rejection:
the process exited 1 after printing "Snapshot generated … Integrity: passed" and writing its
--format json payload, and nothing about the failure reached manifest.json. A hook's own files
were also missing from the ownership inventory, so the next run refused a directory the tool had
written itself, and a console.log inside a hook landed on stdout in the middle of the JSON
payload.

A returned promise is now tracked rather than awaited — this is a synchronous API — and its
rejection is recorded as a warning on the manifest and in the marker, which are re-shipped when it
lands. The marker is written after the hook for the same reason it now includes the hook's files.
stdout is redirected to stderr for the duration of the hook and restored afterwards, so the payload
channel stays pure. Verified: 6 end-to-end tests through the built CLI, including a rejected async
hook that must exit 0 with a parseable payload and a recorded warning.

validate would read any file the manifest named

A manifest is an input, and validate treated it as trustworthy. It joined parts[].filename
onto snapshot/parts with path.join, which accepts ../../../some/file and absolute paths, so a
manifest naming a file outside the snapshot made validate read that file and then report its byte
size — a read primitive, with a size oracle for the file it named, reachable by anyone who can
write a manifest. Part names are bare names by construction, so a name that does not stay inside
snapshot/parts is now a drift error and is never read. explain's text form also re-spells
control and bidi characters in paths, layer names, kind names and part names, because a committed
snapshot.config.* can put a U+202E in a layer name and the terminal renders it.

Verified: 5 new tests — a traversal name, an absolute name, the byte-size oracle specifically
proven closed, a legitimate snapshot still current, and control characters escaped while accented
text passes through untouched.

Also in this release

• --watch reloads a module config. Node's ESM loader caches by URL, so snapshot.config.ts
  (exactly what init writes) was never re-read: an edited config was ignored and a broken one
  was invisible while the run printed passed. The loader now evicts the CJS cache and imports
  with a per-load token. 6 tests.
• --watch never drops an edit. A change during a run, a change inside the post-run quiet
  window, and a source directory whose name matched the outDir basename were all lost silently
  while the manifest kept reporting Integrity: passed. 6 tests plus live repros.
• JWTs are redacted. Only three-segment JWTs with long signatures were masked; two-segment,
  padded, empty-signature and short-signature forms reached the parts verbatim. 8 tests.
• Display hygiene. Control, C1 and bidi characters in file and layer names are escaped wherever
  a path is displayed (parts, tree, navigation, audit, explain), raw bytes are kept in
  manifest.json and file bodies, and the snapshot README and part header say that contents are
  untrusted repository data. 12 tests.
• The hint after init is runnable. It printed npm run snapshot with no package.json, and
  a bare package name on re-runs. 4 tests. The scaffolder now records the tool in
  devDependencies at its own version instead of resolving it from the registry on first run.
  7 tests.
• The linter ignores prose. Content rules matched comments, so documenting why console is
  banned failed the lint that bans it.

validate and report could be talked into saying the state was fine

Four ways the state checks answered confidently and wrongly.

A run killed mid-write left the output directory holding a marker and no manifest, and validate
answered "run generate first" for ever — an instruction that could never resolve, because
generation had already run and failed. The marker records the state, the run id and the start time,
so the message now names them and says whether to wait or regenerate.

A manifest with no config.digest fell back to comparing a handful of fields, so a changed
maxMonolithBytes, layers, kinds or custom secret rule passed as "valid and up to date". A
configuration that cannot be verified is not a match: the absence of the digest is itself reported
as drift.

A file that existed but could not be read was reported as "no longer exist on disk", sending the
reader to look for a file that was right there. Unreadable and deleted are separate findings now,
and the summary says which.

report printed whatever numbers the manifest contained — -5 files included,
Total lines: 1.5, Total size: Infinity MB — and both report and validate accepted
fingerprint: "12345". Counts must be non-negative integers and a fingerprint must be a sha256
digest, or the file is refused as not being a snapshot manifest.

Verified: 17 new tests across tests/snapshot-incomplete.test.js,
tests/validate-drift-classes.test.js and the guards suite, including a real Windows ACL deny for
the unreadable case and a control that an untouched snapshot still reports no drift at all.

validate called a binary file a new file

generate decides "binary" twice: from the extension and the 8 MB head scan, and again on the
bytes it reads, where a control-byte payload or a NUL past the head scan only reveals itself.
validate re-evaluated skipped paths with the cheap pre-read question alone, so a file generate
had just skipped came bac...

4.0.7 — branch names, Azure, and CLI honesty

4.0.7

Wave-2 findings from nine independent critics, one dimension each.

Credentials

• A branch name is user input. git checkout -b "release2/<ghp_…>" copied the token raw into manifest.git.branch, the README and the audit report — a channel that never passed through redaction. Masked now.
• Azure had no rules. DevOps PATs, AD client secrets (…8Q~…) and SAS sig= reached parts and both monoliths verbatim. Added.
• Missing prefixes in otherwise exhaustive alternations: ABIA (AWS bearer), glrt- (GitLab runner), hvb. (Vault batch), 1//0… (GCP refresh), sk_test_ (Stripe test).
• exclude in a config was ignored silently, so a directory the user excluded was still emitted. Accepted as an alias of excludePatterns.

The documented programmatic flow scanned the wrong project

resolveConfig(await loadConfigFile(dir)) ignored dir and generated into process.cwd(), exit 0. The loaded config carries its directory now, so the documented flow scans the project the config describes.

Small lies in the CLI

• --mode was discarded when a command word was present: generate --mode validate wrote a snapshot. A contradiction is exit 2 now.
• --only-artifacts monolith wrote nothing but the marker; naming the monolith now enables it.
• init wrote "snapshot": "codebase-snapshot generate" without installing the package, so npm run snapshot failed with "not recognized". Scripts are npx @ankurhalder/codebase-snapshot … now.

4.0.6 — binary detection and derived layer names

4.0.6

A NUL byte past the first 8 KB. Binary detection sampled 8 KB, so a file with a NUL at byte 9000 was emitted as text into a part. The whole file is scanned now, up to 8 MB, in 64 KB chunks. Verified: a 20 KB file with one NUL at offset 9000 is skipped as binary.

Layer and kind names masked wherever they are derived. Masking config.layers was not enough — the derived values were raw across eleven sinks: manifest.layers keys, files[].layer, part headers, the README parts table, the audit layer table, navigation, report and config.budgets.byLayer keys. The name is now masked where it is assigned, so every derived field inherits it. Verified with a credential-shaped layer name: zero occurrences in any artifact.

4.0.5 — validate notices every settings change

4.0.5

validate now notices every configuration change.

It compared a handful of settings field by field and ignored layers, kinds, secrets.customPatterns, disableDefaultPatterns and maxMonolithBytes. With a config that is not part of the snapshot (include: ["src/**"]), changing classification or redaction rules still reported valid and up to date on a stale snapshot.

The manifest now records a digest of the effective settings, computed from the raw values, and validate compares it. That covers every setting — including ones no field comparison lists — and it works where masked values cannot: two different credential-shaped exclude patterns both mask to the same placeholder.

$ validate   # same config
✅ Snapshot is valid and up to date with disk.

$ validate   # after adding a layer rule
❌ config: configuration: the settings differ from the ones this snapshot was generated with (run generate)

Still open, recorded rather than hidden: 100k files ≈ 67 s (I/O bound); parallel runs with different outDirs can hit the staging lock and exit 1 rather than wait; --watch's quiet window can defer an edit made mid-run; layer/kind names from config are raw in derived fields; binary detection samples 8 KB; the monolith builds before --max-monolith-bytes rejects it.