tether 0.1.0b1

The alpha-exit release, and the first beta. Two security fixes, the engine
bugs an external review found, a hardened backend contract, one migration
for every alpha format (removed at 0.1.0 -- see Deprecated), an
experimental package that is an import boundary with a seamless graduation
path, a narrower undo that reverses only what an operation created,
tether.repo as a package of one module per command family, and -- as an
experimental feature -- a store lifecycle: add --create makes a store
tether owns and gc --delete-stores reclaims it. What still has to run
against real services, and what graduates or goes before 0.1.0, is in
ROADMAP.md.

Upgrade with tether upgrade; move [vcs] git_path/jj_path,
[backends.neon], [backends.ducklake] init_sql, [backends.lakefs], and
any endpoint or credential option out of tether.toml into
.tether/secrets.toml.

Security

• A cloned dataset is untrusted input. The committed tether.toml could
  choose the executables tether runs ([vcs] git_path/jj_path), the
  endpoint credentials are sent to ([backends.neon] api_url,
  storage_options.endpoint, lakeFS client kwargs), SQL to run
  ([backends.ducklake] init_sql), and which environment variable is sent as
  a password ([backends.dolt] password_env). Backends now declare
  SAFE_CONFIG_KEYS; a committed key outside the allowlist -- or an
  endpoint-/credential-shaped key inside an allowed option table -- is
  refused with a message saying where it belongs. Those settings live in the
  new untracked .tether/secrets.toml ([vcs], [backends.<kind>]) or the
  environment (TETHER_GIT, TETHER_JJ). An Iceberg locator's catalog
  table is screened the same way.
• git argument injection. Manifest and state values (ref, at,
  remote, a pin's ref, a sha) reached git positionally without
  --end-of-options, so at = "--output=FILE" made git log write FILE.
  Every git call now passes --end-of-options before its positionals, a
  sha must be hex, and a ref/at/remote/path beginning with - is
  refused at add (ObjectBackend.validate_locator) and at use.

Fixed

• promote moved the trunk bookmark backwards or sideways when the trunk had
  gained commits the bookmark lacked, dropping them off main. A full
  promotion is refused at plan unless the trunk is an ancestor of the
  bookmark's commit (merge or rebase the manifests first, or name keys); at
  apply the trunk is never moved backwards (PromoteReport.trunk_held).
• new on an existing bookmark (reuse) overwrote the branch's fork point
  with its own head, so the next promote saw a spurious "base moved" and
  merged (or refused) where a fast-forward was right. A kept branch keeps its
  fork point, and promote asks the store's own history first
  (ancestor_of), using the recorded fork point only where the backend has
  no DAG.
• Neon put the branch name in the content state, so an untouched fork read
  as modified, commit pinned a child of the working branch for no data
  change, and new on the bookmark demanded --discard. The state's
  branch is now the lineage (the object's source branch when the fork
  descends from it) and the timeline actually read is the volatile
  timeline, so a fork with no writes has the pin's content -- the Lance
  pattern. Neon runs the shared conformance suite and a full Repo lifecycle
  against the API fake; the suite's stability and fork checks compare
  content states (up to VOLATILE_KEYS).
• status spawned two VCS processes per commit in the op log to detect
  drift; vcs_drift now asks once (VcsAdapter.alive_commits).
• Opening a dataset rewrote .tether/.gitignore; Repo.find now writes it
  only when an untracked file that exists is not yet ignored (so an older
  dataset's new secrets.toml never reaches a commit). init and upgrade
  write the full list.
• The checkout lock failed outright when another command held it; it now
  waits up to Repo.LOCK_TIMEOUT (30 s) like the repository lock, and the
  no-fcntl branch is re-entrant.
• A saved gc plan under jj went stale after any snapshot: the digest now
  covers all() ~ working_copies() and binds to the working copy's parent.
• name.2 working refs (a Neon or Lance sibling of a branch that could not be
  reset) are parsed back to their bookmark, and an 8-hex bookmark with a
  suffix is no longer mistaken for a legacy workspace id; new -b refuses a
  bookmark name ending in .<number>.

Deprecated

• The alpha upgrade path. tether.upgrade (the one composed migration
  from any 0.1.0aN format, its history rewriters, and the legacy working-ref
  parsing) ships with the 0.1.0 betas and is removed at 0.1.0. After that, a
  dataset at an alpha version fails at open with a message naming the last
  beta: install tether-vcs==<last beta>, run tether upgrade, reinstall.
  Repo.plan_upgrade / apply_upgrade / upgrade are thin delegates that
  import the package on first use; repo.py and vcs.py read without it.
  tether.migrations is now tether.upgrade.migrations; UpgradeReport
  still imports from tether.

Changed

• tether.repo is a package. The 5,500-line repo.py is split by
  command family -- _core (state, locks, construction, backends, the op
  log, plan verification), _objects (add/remove, set, pull, snapshot and
  status, open, history, verify, diff), _commit, _fork (new, restore),
  _promote, _gc (gc, forget-workspace, abandon), _undo (undo, repair),
  and _reports (the result dataclasses) -- each a mixin over RepoCore,
  assembled into the same public Repo. A pure move: the same 128 methods,
  the same tether.repo exports. Along the way the four workspace-walking
  loops became one _iter_live_workspaces(), and plan_promote reads every
  object's base and working branch in one fan-out instead of two round
  trips per object.
• Round-2 review corrections: committed option tables (storage_options,
  catalog) are screened by a per-backend SAFE_OPTION_KEYS allowlist
  rather than a substring blocklist, so an option tether has never named is
  refused by name; Repo.backend_for reads the class contract via
  backend_class() instead of building a probe instance; tether open
  prints a Neon connection URL with the password redacted unless
  --with-password (never under --json); Icechunk, Delta, and DuckLake
  implement validate_locator (URI scheme, numeric at); Neon takes one
  branch listing per operation and documents the next_xid collision
  between sibling branches; PromoteReport.trunk_moved has its docstring
  back.
• Plan preconditions. What a plan saw is recorded as typed
  Plan.preconditions (manifest_hash, workspace_id, vcs_head,
  history_digest, config_version, ref_absent, ref_head, base_state,
  pin_state, no_new_holders) instead of being re-implemented inline per
  command; one Repo._verify_plan() runs them before any apply_* acts.
  Saved plans are format 2; a format-1 plan (before 0.1.0b1) is refused with
  "re-run the plan". Same semantics for every command, one place to audit
  the drift contract.
• Hygiene: forget-workspace lost its dead delete-branch/keep-branch
  actions and the --force-prune compatibility flag (branches belong to
  bookmarks; gc --prune-bookmarks judges them); Handle.key is documented
  as a display label, not an identity; the docs describe staleness as the
  code enforces it (detected per object, refused workspace-wide) and drop
  the stale --prune-workspaces / --write direct references; the test
  suite finds jj on PATH (or TETHER_TEST_BIN) instead of a hard-coded
  path.
• undo reverses what an operation created, and reports the rest.
  undo new/fork/restore delete the branches the op created and restore
  workspace.toml and the VCS position; a branch the op reset is no longer
  re-pointed to a recorded head (the store may not allow it and the head to
  choose is yours) -- it is reported with its old head and the tool that
  moves it (restore KEY --from REV, new --discard). undo gc restores
  forgotten working refs and listings but no longer recreates deleted
  branches (repair does, from the manifests). undo --to and
  Repo.undo_to are gone: several slips are several undos, newest first.
• tether.experimental. The backends tested only against fakes (Neon,
  lakeFS, Dolt, DuckLake, Iceberg) moved to tether.experimental.backends,
  and the registry layer (export, publish, import) to
  tether.experimental.registry. Nothing users type or import changes: kind
  names, extras, CLI commands, the Repo methods, and the
  tether.ExportBundle / ImportSpec / ImportReport / PublishReport /
  build_bundle / specs_from_rows re-exports are the stable surface.
  It is an import boundary: Repo.export/plan_import/apply_import/
  import_objects are thin delegates to tether.experimental.registry.ops
  imported on first call, the tether.<Symbol> names resolve lazily, and
  the CLI commands are attached from tether.experimental.cli, so
  import tether loads none of it. The alpha-era module paths
  (tether.backends.{neon,lakefs,dolt,ducklake,iceberg}, tether.export,
  tether.registry) are removed without a deprecation window.
  export/publish/import print the experimental note add already
  printed for experimental kinds. Graduating a backend is a file move plus
  MATURITY = "stable" (documented in Extending).
• One migration. tether upgrade brings any alpha dataset to the
  current version in a single step whose parts run on what the dataset shows
  (old-format pins, write = or mtime file states, misplaced manifests or
  relative locators), not on its recorded version. One version write at the
  end, one VCS commit; store renames still fail closed before any history
  rewrite. The plan records its parts.
• Backend contract. Icechunk, Iceberg, Delta, Dolt, Neon, file, and
  DuckLake re-raise their library's exceptions as BackendError
  (wrap_library_errors), so a network blip or a missing ref is a refusal at
  the engine's except TetherError sites, not a traceback from inside a
  client. The ObjectBackend protocol's own fingerprint/pin/fork/open
  bodies raise NotImplementedError instead of returning None.
• Icechunk pin resolves the snapshot before spending a tag name (a missing
  snapshot no longer burns generations); fork leaves a branch already at
  the source alone and wraps reset_branch errors. Git fork skips
  branch -f when the branch is at the source. Delta verify --deep reads
  the version's data files (a vacuumed version is missing). file refuses
  to fingerprint a prefix whose objects report no ETag instead of digesting
  names alone; its listing cache is guarded by the hash-cache lock. Neon pins
  are protected by default with protected_pins = false for the free tier,
  delete_working_ref re-lists to confirm, and NeonHandle keeps the
  password out of repr (redacted_url). Dolt branch_head/resolve no
  longer read a lost connection as "no such ref".
• Conformance suite: the "already at source is left alone" half of the fork
  contract, pin(same id, other state) raises, and open/unpin of a
  missing ref raise BackendError (never a library exception). Iceberg is
  experimental until it runs against a real catalog.

Added

• .tether/secrets.toml also carries per-URI-prefix and per-object
  credentials: profile, role_arn (resolved through boto3 into explicit
  keys), endpoint_url, region, or literal keys. Icechunk passes them to
  s3_storage instead of from_env; file, Delta, and Lance take them as
  storage options. Resolution: object entry, longest URI prefix, kind
  section, then the environment -- an object with no entry behaves as before.
  tether warns when the file is readable by other users and never prints its
  contents. ObjectBackend.configure_secrets/secrets_for are the hooks.

Experimental

The store lifecycle below lives in tether.experimental.lifecycle: the one
operation with no repair path, not yet run against real resources (an S3
prefix, a second clone). add --create and gc --delete-stores say so.
Graduation criteria are in the module docstring.

• Store lifecycle: create and reclaim. tether add KEY LOCATOR --kind KIND --create (Repo.add(..., create=True)) has the backend make an empty store,
  write an owner marker in the store naming the dataset, and record it in a
  repository-wide untracked index (tether-created.jsonl next to the
  repository lock in .jj/repo/ or .git/); the manifest carries
  origin = "created" and status shows (created). On a non-trunk bookmark
  the new store's working branch is forked at once, so no second new is
  needed before the first write. Repo.create(key, kind, locator) is
  add(create=True) plus open -- a throwaway environment's whole setup in
  one line; open itself is unchanged. add --create journals before it
  writes to the store, like every other store-writing command. undo add
  removes the store --create made while it is still empty.
• gc --delete-stores (opt-in: a deleted store has no repair, and gc
  only knows the history this clone has fetched) reclaims created stores
  after the pin and branch decisions. A store is planned for delete-store
  -- last in the plan, behind a store_empty precondition re-checked right
  before the delete -- when its marker names this dataset, no manifest in
  history or in any live checkout's working tree references it, every
  same-dataset branch in it belongs to a bookmark this clone can account for
  (the one it was created on, a live one, or one a new in any live checkout
  made -- a branch of a bookmark this clone never had may be another actor's,
  and the plan then touches nothing in the store), its own pins and branches
  are released by the plan (branches under the --prune-bookmarks rules),
  and the backend confirms nothing else remains. Anything else is a
  keep-store naming what remains; a store already gone is a forget-store.
  GcReport.deleted_stores / kept_stores / forgotten_stores report it;
  undo gc reports a deleted store as irreversible. A clone's manifest never
  authorizes a delete: only the marker and the index in your own repository
  do. gc --store KIND=LOCATOR (plan_gc(stores=[...])) names a created
  store to consider when the creator's clone -- and its index -- is gone.
• gc looks inside the stores this clone has touched: every store a
  checkout forks a branch in or pins is recorded in tether-touched.jsonl
  beside the created index, and a plain gc releases this dataset's dead
  refs there once no manifest names the store any more (an abandoned bookmark
  took the manifest with it). Before, those refs stayed forever and kept the
  store's creator from ever reclaiming it. An entry is dropped
  (forget-touched) once nothing of the dataset's is left. memory accepts
  the positional uri locator like other kinds.
• Backend contract: Capability.CREATE with create(locator, *, owner),
  owner(locator), is_ref_empty(locator, *, ignoring=()) (None means
  keep; a working area must hold no uncommitted content), and
  delete_store(locator) (re-checks both, removes only the store's own
  layout); implemented for memory, icechunk (repository metadata marker;
  refuses a non-empty path or prefix; deletes only Icechunk's top-level names,
  1.x and 2.x layouts), git (init with one fixed empty root commit shared by
  every created repository; git config tether.owner; a dirty working tree or
  an amended root is not empty), and file
  (a local directory with a .tether-owner marker the directory walk skips).
  Lance, Delta, and Iceberg need a schema and do not declare it. The
  conformance suite walks the lifecycle for harnesses that provide
  fresh_locator().