4.1.0 — encoding honesty, ownership evidence, discovery modes and a diff that reports payload changes

A single release carrying the whole night's work: honesty about encodings, ownership and state, no silent data loss, bounded memory, two hard crashes fixed, a new --discovery mode, and a diff that reports when the snapshot handed to a model changed even though the sources did not.

Full detail below. Upgrading is additive — one new skip reason, two new optional manifest fields and one new config setting; nothing removed, nothing required.

A UTF-16 source file was invisible, and a Latin-1 one was quietly mangled

Two encoding decisions were silent, and one was wrong in both directions. generate decided
"binary" from a NUL in the first 8 MB and then decoded with buffer.toString("utf8"), so a UTF-16
source file — which is half NUL bytes — was skipped as binary and vanished from the snapshot,
while a Latin-1 file was emitted with U+FFFD in place of every non-ASCII byte, and validate then
reported the snapshot as current because the hash covers the raw bytes.

Decoding is now explicit. A BOM, or the NUL padding of a BOM-less UTF-16 file, is recognised and
the file is transcoded; a file that is not valid UTF-8 is kept, decoded lossily, and labelled
encoding: "utf-8-lossy" with the count of undecodable bytes in warnings; a NUL anywhere in the
file — including past the 8 MB head scan, which used to be the end of the check — marks it binary
and skipped[] records the offset. files[].encoding is present only when a file was not plain
UTF-8, so its absence keeps meaning "clean UTF-8". explain answers the same question the same
way. Verified: 12 new tests covering both endiannesses, BOM-less detection, five classes of
invalid sequence, a NUL at byte 8388608, and hash stability across two runs.

A file the tool did not write could be deleted without a word

Ownership checking had a hole with a nice name. .gitkeep and .gitignore were treated as
placeholders and subtracted from the set of entries belonging to somebody else, so a snapshot
directory this generator owned and that a user then added a .gitignore to was replaced — atomically,
exit 0, manifest.warnings empty — and the file was gone. A directory holding nothing but
.gitkeep, or nothing but a .gitignore (*, !.gitignore), was replaced without any ownership
evidence at all, because the placeholder shortcut fired before the marker was ever read.

Both are files a user creates; the generator never writes either into an output directory. They are
foreign entries now, so the run refuses and names them, exactly as it already did for notes.md.
.DS_Store remains the only tolerated entry, because Finder writes it and it carries no intent.
Verified: 6 new tests — owned directory plus .gitignore, bare .gitkeep, bare .gitignore, the
refusal naming the entry, .DS_Store still discarded, and a genuine snapshot still replaced.

A directory named snapshot disappeared, and the manifest said nothing

The output directory was excluded from the scan by name, at every depth: snapshot,
snapshot.tmp and snapshot.backup-* were dropped before the branch that records a skipped[]
entry, so a directory the user committed under one of those names was missing from the snapshot
with no record anywhere — the exact omission skipped[] exists to prevent, and it contradicted
the documented promise that every discovered path is accounted for. generate and explain also
disagreed: one dropped the path, the other called it not-discovered.

Exclusion is now evidence-based. A directory is this tool's output when it carries the
.codebase-snapshot marker, and only the exact outDir/tmpDir paths a run is about to write are
excluded by name — so a first run still does not record its own output (two identical runs keep
producing identical manifests) while a user's directory with the same name is scanned like any
other. snapshot and snapshot.tmp also left the built-in ignore list, where they had been
second-layer protection for the same mistake. Verified: 5 new tests, including the same fixture
through the git and the walking discovery paths, a full generate that now lists
snapshot/keep.md, and the determinism guard that the second run does not record the first.

A hook could kill a run that had already succeeded

hooks.afterGenerate ran after the snapshot was swapped into place. A hook that returned a
rejected promise was never awaited and never caught, so Node turned it into an unhandled rejection:
the process exited 1 after printing "Snapshot generated … Integrity: passed" and writing its
--format json payload, and nothing about the failure reached manifest.json. A hook's own files
were also missing from the ownership inventory, so the next run refused a directory the tool had
written itself, and a console.log inside a hook landed on stdout in the middle of the JSON
payload.

A returned promise is now tracked rather than awaited — this is a synchronous API — and its
rejection is recorded as a warning on the manifest and in the marker, which are re-shipped when it
lands. The marker is written after the hook for the same reason it now includes the hook's files.
stdout is redirected to stderr for the duration of the hook and restored afterwards, so the payload
channel stays pure. Verified: 6 end-to-end tests through the built CLI, including a rejected async
hook that must exit 0 with a parseable payload and a recorded warning.

validate would read any file the manifest named

A manifest is an input, and validate treated it as trustworthy. It joined parts[].filename
onto snapshot/parts with path.join, which accepts ../../../some/file and absolute paths, so a
manifest naming a file outside the snapshot made validate read that file and then report its byte
size — a read primitive, with a size oracle for the file it named, reachable by anyone who can
write a manifest. Part names are bare names by construction, so a name that does not stay inside
snapshot/parts is now a drift error and is never read. explain's text form also re-spells
control and bidi characters in paths, layer names, kind names and part names, because a committed
snapshot.config.* can put a U+202E in a layer name and the terminal renders it.

Verified: 5 new tests — a traversal name, an absolute name, the byte-size oracle specifically
proven closed, a legitimate snapshot still current, and control characters escaped while accented
text passes through untouched.

Also in this release

• --watch reloads a module config. Node's ESM loader caches by URL, so snapshot.config.ts
  (exactly what init writes) was never re-read: an edited config was ignored and a broken one
  was invisible while the run printed passed. The loader now evicts the CJS cache and imports
  with a per-load token. 6 tests.
• --watch never drops an edit. A change during a run, a change inside the post-run quiet
  window, and a source directory whose name matched the outDir basename were all lost silently
  while the manifest kept reporting Integrity: passed. 6 tests plus live repros.
• JWTs are redacted. Only three-segment JWTs with long signatures were masked; two-segment,
  padded, empty-signature and short-signature forms reached the parts verbatim. 8 tests.
• Display hygiene. Control, C1 and bidi characters in file and layer names are escaped wherever
  a path is displayed (parts, tree, navigation, audit, explain), raw bytes are kept in
  manifest.json and file bodies, and the snapshot README and part header say that contents are
  untrusted repository data. 12 tests.
• The hint after init is runnable. It printed npm run snapshot with no package.json, and
  a bare package name on re-runs. 4 tests. The scaffolder now records the tool in
  devDependencies at its own version instead of resolving it from the registry on first run.
  7 tests.
• The linter ignores prose. Content rules matched comments, so documenting why console is
  banned failed the lint that bans it.

validate and report could be talked into saying the state was fine

Four ways the state checks answered confidently and wrongly.

A run killed mid-write left the output directory holding a marker and no manifest, and validate
answered "run generate first" for ever — an instruction that could never resolve, because
generation had already run and failed. The marker records the state, the run id and the start time,
so the message now names them and says whether to wait or regenerate.

A manifest with no config.digest fell back to comparing a handful of fields, so a changed
maxMonolithBytes, layers, kinds or custom secret rule passed as "valid and up to date". A
configuration that cannot be verified is not a match: the absence of the digest is itself reported
as drift.

A file that existed but could not be read was reported as "no longer exist on disk", sending the
reader to look for a file that was right there. Unreadable and deleted are separate findings now,
and the summary says which.

report printed whatever numbers the manifest contained — -5 files included,
Total lines: 1.5, Total size: Infinity MB — and both report and validate accepted
fingerprint: "12345". Counts must be non-negative integers and a fingerprint must be a sha256
digest, or the file is refused as not being a snapshot manifest.

Verified: 17 new tests across tests/snapshot-incomplete.test.js,
tests/validate-drift-classes.test.js and the guards suite, including a real Windows ACL deny for
the unreadable case and a control that an untouched snapshot still reports no drift at all.

validate called a binary file a new file

generate decides "binary" twice: from the extension and the 8 MB head scan, and again on the
bytes it reads, where a control-byte payload or a NUL past the head scan only reveals itself.
validate re-evaluated skipped paths with the cheap pre-read question alone, so a file generate
had just skipped came back as new file and the first validate after a generate failed on an
untouched tree. Both commands decide the same way now.

--discovery chooses how files are found

A .git directory that exists but cannot be listed made discovery refuse, and there was no
override — the only way forward was to repair the repository or copy the tree out of it. The
refusal also justified itself with a gap that no longer exists (a walk does honour nested
.gitignore files now), so it names the differences that remain: info/exclude, the global
excludes file, tracked files that a .gitignore now matches, and a submodule as one entry.

--discovery auto|git|walk (config discovery) makes that choice explicit. auto keeps the old
behaviour, git requires git, walk scans the tree and reads .gitignore directly. The mode is
validated, scoped to the commands that scan, published in the contract, written to the init
template, and recorded in manifest.config, so changing it is config drift. 10 tests.

A workspace is classified by what its packages are

Detection read only the repository root, so a Next.js app under apps/web in a pnpm workspace
detected as monorepo and a docs repository whose markdown lived under docs/ detected as
generic — both contradicting the published preset table. Detection now probes a bounded number
of workspace packages and markdown directories, so a workspace is classified by the frameworks its
packages use. The bounds keep it honest: a bare apps/ without a workspace declaration does not
trigger it, and a framework two levels down still detects as monorepo. 8 tests.

skill --install could write outside the repository

The installer built its three targets from --cwd and wrote them without checking where they
landed, so a pre-existing junction (.claude itself, or .claude/skills/codebase-snapshot) or a
hard-linked AGENTS.md made it write outside the repository. Targets are resolved and refused
when they leave it. The section merge also repairs malformed marker layouts — duplicated blocks,
an inverted pair, a missing end marker — instead of appending another one, while a well-formed
file stays byte-identical on a rerun. 11 tests.

diff saw no change when the payload changed

diff compared relativePath plus the source hash, so two snapshots of identical sources with
different --max-part-bytes or --large-file-bytes — a 4 KB preview against a 36 KB full body,
275 KB of parts against 307 KB — reported "0 added / 0 removed / 0 changed". The per-file answer
stays the same, and a payload block now says the configuration differs and why (the config digest,
the generator version, part and layer totals), which is what "what changed in the context we hand
to the model?" was asking. 5 tests.

---