tether 0.1.0a10

0.1.0a9 was tagged in history but never published; a10 is the first release
carrying both sets of changes. Datasets created with a8 need tether upgrade
(v3 and v4 migrations; working tree only, no history rewrite).

Added

• The op log is a journal. Store-writing commands (commit, pull,
  new, the lazy fork, gc, promote, restore, repair) write their
  entry -- plan and what undo needs -- and sync it before the first side
  effect, then a completion mark with the result. An interrupted run leaves an
  INCOMPLETE entry: ops flags it, undo skips it (and says why when
  named), repair --dry-run lists it with what gc will collect
  (Repo.incomplete_ops()).
• One writer per checkout. Writing commands hold .tether/lock (flock,
  re-entrant per Repo) so two tether processes cannot interleave journal
  entries and workspace.toml writes.
• Branch scope. ObjectBackend.branch_scope(locator) names the resource
  that owns branches and ref_namespace(locator) the one whose pins
  list_pins returns (both default to the canonical identity; Neon: the
  project, Iceberg: the table). new forks one branch per scope under a
  bookmark -- the first member forks, later members share it, a member that
  pins a different state of the same branch is refused -- and gc collects
  the pins every object references per namespace.
• Pin.created (runtime-only): whether pin() made the ref or found it
  already carrying the state. The conformance suite checks both answers.
• DiffEntry.why: which of state, pin, locator, policy differ. A
  locator- or policy-only change is changed (CLI: [policy changed; same state]; --json carries why).
• tether backends lists kinds with maturity, tier, and capabilities;
  tether --version. Backends declare MATURITY (stable: full lifecycle
  against the real system in CI; experimental: tested against a fake of a
  network service -- neon, lakefs, ducklake, dolt); add notes an
  experimental kind.
• ObjectBackend.LOCAL_PATH_KEYS: locator keys that may hold a local path.
  The git backend runs the shared conformance suite.
• ObjectBackend.state_addressable(locator, state): whether a particular
  recorded state can be reopened (the file backend says no for a remote
  object without a version id); commit records such a state
  recoverable = false and says why in the plan.
• VcsAdapter.history_digest(): a digest of every visible commit id, across
  workspaces and bookmarks; gc plans bind to it.
• A repository-wide lock. commit, pull, gc, undo, and abandon
  hold tether.lock in the store every checkout shares
  (VcsAdapter.shared_dir(): git's common dir, jj's repo dir), waiting up to
  Repo.REPO_LOCK_TIMEOUT, so a gc in one workspace cannot race a commit in
  another between deciding a pin is unreferenced and releasing it.
• Progress records. Every side effect of a journaled operation appends a
  record (mark_progress; OpEntry.progress): each pin and the VCS commit of
  a commit, each fork of new/restore, each unpin and deletion of gc,
  each system a promote lands, each repin/refork of repair. repair --dry-run lists what an incomplete operation got done, how many actions
  were planned, and the re-run contract (running the command again finishes
  what is left).

Changed

• Config v4 (tether upgrade; working tree only, no history rewrite).
  Manifest paths append .toml to the key's last segment instead of
  replacing its suffix, so foo and foo.bar no longer share
  objects/foo.toml (the migration moves misplaced files; a collision that
  already destroyed a manifest is reported). Keys are validated: no empty
  segments, ./.., leading slash, or control characters. Relative local
  paths in locators are resolved against the dataset root (the migration
  rewrites them); add and import resolve a relative path against the
  caller's directory and store it absolute.
• A pin is verified before it is read or forked. open --rev, new from
  a commit, and promote --rev check the pin against the manifest's state:
  a deleted pin falls back to the recorded state where the backend can
  address it; a moved pin raises PinDriftError (a refuse in a promote
  plan). repair never overwrites a drifted pin.
• Destructive steps re-check the ref they act on. Plans record the head
  of every branch they delete or reset; apply reads it again immediately
  before the step and stops with StalePlanError when it moved: gc
delete-branch (gc plans are also bound to the VCS head and manifest
  hash), new's reuse/reset (plus a re-run of the bookmark-holder guard and
  a same-workspace check), restore's reset, promote's source, repair's
  refork (the branch must still be missing). A lazy fork resets an existing
  branch only onto the head new reviewed (workspace.toml
pending_resets), reuses a branch already at the pin or one a scope
  sibling writes through, and otherwise refuses. gc plans are bound to the
  digest of all visible history (a commit made in another workspace can
  reference a pin the plan would release) and check every branch head before
  the first action, so a stale plan does nothing at all; promote --rev
  verifies a pin source still names the reviewed state; a fork new planned
  as fresh is re-checked for absence at apply, and new refuses outright
  when the backend cannot list branches (unknown is not absent).
• commit compensates as a unit. A failure after the pins -- manifest
  write, listing, VCS commit -- releases only the pins this commit created
  (never a reused one), restores the manifests and listings it wrote, and
  journals the attempt as failed and rolled back. If the VCS commit landed
  before the adapter raised, nothing is rolled back: history names the pins,
  so the operation completes as a commit that succeeded and the trailing
  error is surfaced (failed_after_commit).
• restore and promote are closed over the branch scope. Restoring one
  of several keys that write through one branch is refused (the message names
  the siblings to include); with all named, the branch is reset once and the
  siblings share it. promote fast-forwards a shared branch once instead of
  once per key.
• The writer lock also covers add, remove, set, import, abandon,
  forget-workspace, and upgrade; undo journals before it acts (a refused
  undo is recorded as failed). Taking the lock re-reads workspace.toml and
  the manifests, so a long-lived Repo never writes the state it loaded at
  construction over what another process wrote since; snapshot writes its
  cache under the lock.
• undo completes atomically: the undone mark rides in the done record (one
  append), and a handler refusal that touched nothing ends the entry as a
  failed attempt rather than leaving it started.
• promote KEY... refuses a subset that leaves unnamed siblings whose base
  branch the write would move, whatever the source (a working ref, or a pin
  by --rev), as restore does.
• promote fast-forwards and merges from the state the plan reviewed, not
  the source ref's current head: what lands is what was shown, whatever the
  timing. ObjectBackend.merge takes str | Pin | State like promote
  (git, lakeFS, Dolt, and memory resolve a state to its commit). The inline
  convenience methods (commit, new, promote, restore, gc, import)
  plan and apply under one checkout lock, and new/promote/restore/
  import re-verify at apply (commit does not need to: a pin names the
  captured state). The CLI's immediate tether commit goes through
  Repo.commit too; only --dry-run/--plan build a separate plan.
• gc plans take the history digest before walking history, so a commit
  landing during the walk stales the plan instead of slipping between the
  references and the digest.
• restore checks every head before the first reset and writes the
  workspace after each one (with the siblings sharing the branch), so a kill
  between two resets leaves each reset branch described as such.
• apply_commit turns a planned key that is no longer registered into
  StalePlanError (rolling back the pins it made before reaching it) rather
  than a KeyError.
• promote --rev's scope closure and base-head check work from the kind and
  locator the plan captured, so an object removed from the working tree since
  the revision still refuses an unnamed sibling and a base that moved.
• snapshot and pull run whole under the checkout lock: which refs to read
  is decided from the workspace as it is on disk, not from the one a
  long-lived Repo loaded.
• new writes workspace.toml -- bookmark set, every fork pending with the
  reset it agreed to -- right after moving the VCS and before the first store
  write, so a process killed in the fork fan-out leaves exactly a lazy new;
  a fork that fails in the fan-out stays pending instead of being forgotten.
• gc: a branch that moved after the preflight (a race, not a stale plan)
  is kept and reported while the rest of the plan finishes; a --force-prune
  plan that could not read a head refuses at apply if the head reads now.
• Pre-v4 manifests read from history resolve relative local paths against
  the dataset root, the rule the migration applies to the working tree.
• promote's guarantee is stated as it is: a bookmark is planned whole or
  not at all; once applying, each system's fast-forward stands on its own.
• set --pin record on a pinned object takes effect at the next commit (the
  pin is dropped; gc releases the tag once no commit names it); --pin native creates one again. Before, the "unchanged" shortcut kept the pin.
• file: --file versioned makes only a single remote object Addressable;
  a recorded object state without a version id (unversioned bucket) is
  refused by open and reported by verify. The content-hash cache keys on
  ctime_ns too and re-reads entries hashed within the same second on
  filesystems with whole-second mtimes.
• Backends: unpin and delete_working_ref (git, lance, lakefs, icechunk,
  neon) raise when the ref is still there afterwards instead of reporting
  success; Lance says why a tagged branch stays. delta/iceberg read-only
  open of an object registered at a version sits there, not at the head;
  iceberg open(Pin) raises for a missing tag. dolt.ancestor_of returns
  unknown when the log is truncated. lakefs fork resets a branch with
  staged uncommitted objects. memory.history walks the head's parents.
  neon follows branch-list pagination, lifts the protection before deleting
  a pin, and no longer calls a pin drifted for the read-only endpoint tether
  attaches to serve open.
• publish.yml runs lint, ty, and the suite, checks the tag against the
  package version, and smoke-tests the built wheel before uv publish.

Fixed

• commit rolled back pins it had not created: pin() is idempotent, so a
  pin an earlier commit (or a sibling key on the same system) made was
  released when a later pin in the same commit failed.
• open --rev, new, and promote --rev trusted a pin's native ref; a tag
  moved by hand returned the wrong data under a commit's name.
• Two keys on one native branch space (a Neon project, one memory system)
  each forked the shared bookmark branch from their own pin: opening the
  second reset the first's writes. gc grouped references per identity while
  list_pins lists a whole project, so one object's sweep released its
  neighbours' pins.
• A saved gc plan deleted a branch that had gained writes since planning;
  a promote landed a source that moved after review.
• Repo.diff reported an object unchanged when only its locator or policy
  differed.
• Relative paths in locators were resolved against each command's working
  directory, so the same manifest addressed different files from different
  directories. The git backend left the CLI's positional uri relative.
• The v4 migration rewrote a relative locator before moving a misplaced
  manifest, leaving two files for one key; it recorded v4 after reporting a
  manifest collision (it now stops, like v3 on a failed re-fingerprint); and
  a collision found half-way left earlier moves in a dirty tree that blocked
  the rerun (every destination is checked before anything moves).
• snapshot from a stale Repo cached one bookmark's states under another's
  name: it chose which refs to read before the lock refreshed the workspace.
• snapshot from a Repo constructed before another process moved the
  checkout to a bookmark rewrote workspace.toml with the old bookmark and
  refs.
• undo's two-append completion could leave a finished undo whose target
  still counted as undoable.
• --file versioned recorded a state with no version id as recoverable
  although open could not read it back.
• The wheel smoke test in publish.yml installed the wheel without the cli
  extra the entry point needs.

[0.1.0a9] - 2026-09-11

Added

• Bookmark-shaped branches. The dataset's jj/git bookmarks and the
  stores' branches are now one shape. The trunk bookmark (main;
  [vcs] trunk) stands for every object's upstream branch (locator.branch):
  working on it writes there. Any other bookmark stands for one branch per
  Forkable system, named after it -- tether.ws.<dataset>.<bookmark> --
  forked from the pins of the commit it started at. A working copy on no
  bookmark is read-only. In detail:
  • tether new -b NAME [REV] creates a bookmark and its branches (lazily, as
    before); tether new NAME joins one; tether new REV takes the bookmark
    at that commit or goes read-only. A bookmark another live checkout works
    on is refused unless --shared. init creates the trunk bookmark (jj) or
    adopts HEAD's branch (git) and starts there. WorkspaceState.bookmark.
  • commit moves the bookmark onto the new commit -- in the same jj
    operation, so jj undo takes both back -- and refuses when the VCS
    working copy has left the bookmark, or (jj) when the bookmark is behind
    the working copy's parent, the one position from which jj can carry it
    along in that operation; tether new NAME --keep puts the working copy
    back without touching branches. undo of a new -b deletes the
    bookmark it made. abandon moves a bookmark off a dropped commit to the
    nearest kept one instead of losing it (jj deletes them).
  • tether pull [BOOKMARK] is the fetch: it reads the heads of the bookmark's
    branches -- on the trunk every upstream branch and every branch-less
    object -- pins what moved, and commits it on the bookmark, which moves.
    Nothing moved: no commit. PullReport (bookmark, committed,
    unchanged, skipped, vcs_commit, pinned). Replaces the held
    [pulled] workspace state, the pulled status label, commit --pull,
    and [commit] pull.
  • promote also moves the trunk bookmark to the bookmark's commit when
    every object fast-forwarded and nothing was refused
    (PromoteReport.trunk_moved); after a merge, commit and promote again.
  • gc --prune-bookmarks (was --prune-workspaces) judges the branches of
    bookmarks the VCS no longer has and no live checkout works on, legacy
    per-workspace branches, and this bookmark's unused ones, with the same
    verdicts; --keep-bookmark NAME. forget-workspace removes state files
    and forgets the checkout only: branches belong to bookmarks.
  • status names the bookmark (on trunk bookmark main; on no bookmark: read-only) and warns when the VCS deleted, renamed, moved, or left it,
    with what to do (Repo.bookmark_drift, StatusReport.bookmark,
    .trunk, .bookmark_drift).
  • VcsAdapter gains bookmarks, bookmark_set, bookmark_delete,
    current_bookmarks, new_bookmark, is_ancestor, and
    commit(advance=); ObjectBackend.base_branch; working_ref_name(dataset, bookmark), working_ref_bookmark, bookmark_slug.
• tether set KEY... | --all [--file] [--pin] changes a registered
  object's policy in place (Repo.set_policy, SetReport): manifest-only,
  logged, undoable. Until now this took remove + add (losing the
  committed state) or an import from a registry.
• A Caveats and Performance guide, holding what the README used to: the
  limits of the model (per-system promotion, no cross-system atomicity, what
  undo can and cannot do, pin-then-commit ordering, lazy forks, storage
  cost, secrets), the per-backend caveats, and the performance notes. The
  backends guide gains the content-diff table. The README is a third of its
  former length: capability table, prior art, install, a short quickstart,
  layout, non-goals, development.
• A Use Cases guide: seven scenarios -- reproduce an analysis months
  later; reprocess on a branch then land or discard it; A/B two candidates
  and keep one; catch drift nightly without a watcher; publish history to a
  registry; keep the data bill down; recover -- each as the commands you run,
  what they guarantee, and the guide that explains the mechanics. The README
  lists them. Later guides are renumbered (07-cli ... 11-extending).
  The scenarios are one running story, with jj log after each step, and
  the whole page is executed, not written: tests/test_use_cases.py
  runs every command against local Icechunk, Lance, file, and git objects
  (and the ephemeral Postgres the publish tests use; without one the story
  stops before section 7) and writes the command and output blocks the guide
  includes (user_guide/_generated/06/), with per-run ids replaced by stable
  stand-ins and the operation log on a fixed clock; the test fails when a
  fresh run differs, and TETHER_UPDATE_DOCS=1 rewrites the files. Running
  it found the fixes below.

Changed

• promote lands a bookmark whole or not at all. It used to fast-forward
  the systems it could while refusing the others -- the executed guide showed
  Icechunk's main moving while Lance was refused, so readers of main saw
  new labels with old features, and the trunk bookmark could not follow.
  Now, when any object is refused and no keys were named, the rest are
  held (PromoteReport.held, printed with what each would have done) and
  nothing is written. tether promote KEY... lands a subset on purpose; the
  trunk bookmark never moves for a subset, since the rest has not landed.
  Exit status is still 1 on refusals.
• [new] auto_fork is gone. It re-ran new after every commit to give
  jj's "fresh working copy" rhythm; since new reuses a branch that already
  sits at the pin, it had stopped doing anything. commit leaves working
  branches where they are, and the docs now say so (Concepts: "Where tether
  is not jj").
• The write policy is gone. Whether writes fork a branch or land on
  the upstream branch was write = fork | direct (formerly track) per
  object; it is now which bookmark the working copy is on. Policy is file
  and pin; --write, set --write, [defaults] write, the policy_write
  registry column, and WriteMode are removed. A manifest carrying write
  is read and ignored, and the v3 migration (tether upgrade) drops the line
  from the working tree alongside the content-hash re-fingerprint. Neon
  databases that must receive writes on main are written on the trunk
  bookmark.
• tether status is local by default. It shows the last snapshot of each
  object's state with its age (states as fingerprinted 2h ago; --snapshot to refresh) and contacts nothing, so it is cheap enough to run as often as
  jj status. --snapshot fans out and fingerprints first; a workspace with
  no snapshot yet always does. [snapshot] auto now defaults to false
  (true restores fingerprinting on every status; --no-snapshot wins).
  verify always fingerprints regardless of the setting; commit fingerprints
  the bookmark's branches.
  StatusReport gains fresh and snapshot_at, and the JSON output the same.
• Local files are fingerprinted by content hash, not mtime. A file's state
  is {size, sha256} and a directory's digest is over its files' sha256s, so
  touch, cp, a fresh checkout, or an rsync no longer read as drift (an
  error under the default immutable policy) or mint a new state. Hashes are
  cached by (size, mtime_ns, inode) in the untracked
  .tether/cache/file-hashes.json -- the git/DVC pattern -- so the first
  fingerprint of a tree reads every file and later ones read only what
  changed. ObjectBackend.configure_cache(dir) is the hook the engine calls
  so a backend can keep such scratch state. Listing tokens for local
  directories are sha256s (were size:mtime_ns). Breaking: manifests
  from earlier alphas hold mtime-based states that would read as drift -- an
  error under the default immutable policy -- so [tether] version is now
  3 and tether upgrade (v3 migration) re-fingerprints every local file
  object in the working tree and rewrites its manifest; remote objects and
  history are untouched.

Fixed

• ForgetWorkspaceReport was listed in tether.__all__ but never imported,
  which broke the docs build; a test now checks __all__ against the module.
• tether diff REV (one revision) compared REV with nothing and reported
  every object removed; it now compares REV with the working tree, as its
  help always said.
• tether diff --content between two Icechunk snapshots on different
  branches failed with Icechunk's "ancestry doesn't include" error, since it
  diffs along one line of history. The backend now finds the snapshot the two
  diverged from and reports what either side changed since it (a node only
  one side touched keeps that side's change, inverted for the from side;
  one both touched is modified), with a note naming the base.
• Fingerprinting several local file objects at once (commit, status --snapshot, pull fan out concurrently) could fail with No such file or directory: .file-hashes.json.tmp: the shared content-hash cache was written
  by two threads through one temp file. The cache is now locked around its
  table and its save; hashing itself still runs in parallel.
• A deleted Icechunk pin could never come back: Icechunk keeps a tombstone
  for every deleted tag and never lets the name be reused, and pin ids are
  content-addressed, so repair crashed with RefNotFoundError and any later
  commit recording that same snapshot for that object failed too. The pin id
  stays content-addressed; the ref moves on: the Icechunk backend walks
  generations -- tether.<id>, tether.<id>.2, .3, ... -- past tombstones
  (a name carrying a different snapshot is still an error), and verify,
  open, fork, and promote resolve a pin through the newest live
  generation when the manifest's ref is gone, so manifests and history stay
  as they are (verify says pinned as tether.<id>.2). list_pins folds
  generations onto their id and unpin clears them all, so gc counts one
  pin. Until a repair, Repo.open (at a revision or at the object's
  position) and new's fork fall back to the recorded state when the pin's
  native ref is gone and the backend is Addressable.
• tether new -b NEW from a commit on bookmark OLD handed NEW the store
  branches of OLD wherever they sat exactly at the pin ("already at pin;
  kept"), so two bookmarks shared a branch and NEW's writes landed on
  OLD's -- a leftover of the per-workspace naming. new now looks only at
  the branch named after the bookmark it is starting: kept if it exists at
  the pin, reset (recorded for undo) if it exists elsewhere, refused if it
  holds writes since the last commit -- or if its head cannot be read, since
  a fork would then reset it blind -- and forked otherwise. Another
  bookmark's branch is never touched.