Skip to content

v0.8.0 — witness export --all: your backups were missing your lenses

Latest

Choose a tag to compare

@IngTian IngTian released this 24 Aug 18:58
6fe528d

If you have ever run witness export as a backup, that backup is incomplete — please re-take it with --all. This release exists because of that, and it is the reason for a minor rather than a patch bump.

witness export was only backing up the database

witness export <file> writes a consistent snapshot of witness.db, and the README called that "back up the archive". But only one of the four things in your data directory is the database. Three sat beside it and went unbacked-up, silently, for the whole life of the command:

in --all in a plain export recoverable without a backup?
witness.db — raw turns, observations, facets yes yes no
config.toml — enabled lenses, runner, models yes no reconfigure by hand; until then distillation is silently off
lenses/ — your lens definitions yes no no — prompt text is in no database
profile/ — the narrative yes no rebuilt by a review, except a hand-edited unified.md

lenses/ is the one that hurts. A lens witness ships re-seeds itself from prompts/; a lens you wrote cannot — the prompt text exists nowhere in the database, so restoring from a snapshot gave you back every observation it had ever produced and none of the prompt that produced them. If you distill a non-person corpus, that authored lens is the most valuable thing in your install.

profile/unified.md got sharper in 0.7.6, which made hand-editing it the supported way to shape the cross-lens portrait. There is no second copy of that edit.

The fix

witness export --all ~/Dropbox/witness-backup --force

--all makes <path> a directory and writes all four pieces. Left out on purpose: witness.log (diagnostics), runtime/ (OpenCode's disposable private runtime, rebuilt on the next open, and large), the WAL sidecars (folded in by VACUUM INTO — copying them live is the corruption mode export exists to avoid), and the lock files.

Restoring needs no new command. The snapshot directory has the same layout as your data directory, so it is itself a working archive:

WITNESS_HOME=~/Dropbox/witness-backup witness status   # just works

Or copy its contents into your data dir with witness stopped. That property is what kept this small — no import, no reassembly, nothing to document beyond one env var.

Either form still runs safely while the background worker is writing. No need to stop it.

Backups fail silently, so the failure modes are the feature

  • Your previous backup survives a failed export. Everything stages into a sibling directory and swaps in only once complete, with any prior snapshot moved aside rather than deleted. A cron export --all --force that dies halfway leaves yesterday's backup intact — verified for a failure before the swap and during it.
  • It refuses to export onto your live data directory. Compared by inode identity, so a symlink or a different spelling of the same path is caught too. With --force the old code path would have moved your real archive aside and then deleted it, possibly mid-write.
  • Symlinks in the lens registry are skipped, not followed, so a planted link can't make a backup reach outside the archive.
  • export <dir> now points you at --all instead of failing with "directory not empty", and export --all foo.db is rejected rather than creating a directory named foo.db.
  • --json reports what was actually captured, so a scripted backup can verify itself rather than trusting an exit code.

Every one of those was confirmed by reverting the production line and watching the test go red — 14 mutations. Two of the tests were vacuous on the first pass and were fixed: the "keeps the previous backup" test only exercised the pre-swap path, and the symlink check had no symlink to skip.

Exercised end to end against a real 363-session / 9158-observation archive while the worker was running: PRAGMA integrity_check ok, identical row counts, and the config, both lenses and the portrait all read back from the snapshot used as a data directory.

Nothing else changed

No engine, schema, capture, or distillation changes. Export's own documentation claimed the snapshot contained "config", which is how the gap stayed invisible; that is corrected.

Install / upgrade

macOS · Linux — download the binary for your platform below, then:

chmod +x witness-v0.8.0-darwin-arm64
./witness-v0.8.0-darwin-arm64 doctor        # confirm it runs

Replacing an existing install: put it where your current witness lives and re-run witness wire claude (or wire opencode) only if you moved the path. Your archive is untouched by an upgrade.

npm (OpenCode)npm i -g @witness-ai/opencode@0.8.0

Windows — download witness-v0.8.0-windows-amd64.zip (or -arm64), unzip it, and run from the unpacked witness\ folder:

witness.exe wire claude       REM or: wire opencode

The zip is self-contained (witness.exe + prompts\ + the embedding model) and the binary locates those relative to itself, so keep the folder together. Replacing an older install: unzip over the existing witness\ folder and re-run witness.exe wire <editor>; install copies the binary and assets into %LOCALAPPDATA%\witness and writes atomically, so re-running is safe.

Verify any download against SHA256SUMS:

shasum -a 256 -c SHA256SUMS --ignore-missing