Skip to content

Releases: elyall/tether

tether 0.1.0b6

tether 0.1.0b6 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 01 Oct 17:17

Added

  • tether init --dataset-id ID uses that id instead of a fresh one; anything
    but 8 lowercase hex characters is refused.
  • tether restore KEY... --at REF starts the bookmark's branch as a copy of
    a native branch, tag or state id. REF itself is never moved.
  • tether new --adopt takes the bookmark's existing store branches as they
    are, uncommitted writes included, instead of resetting them. A saved plan
    binds to each branch still existing and being the newest generation, not
    to its head.
  • tether recover lists tether's refs in each store by dataset id and prints
    the steps that take back a dataset whose repository was lost.
  • tether repair KEY... (Repo.repair(keys)) rebuilds only those objects'
    pins and branches, selected from every key history has had with
    --all-history. A saved plan records its selection under its digest.

Changed

  • A saved plan (--plan FILE) records the tether version that made it, and
    --from-plan applies it only under that version. Plans saved by any other
    version, including every earlier release, are refused: re-run the plan.
  • tether recover prints each step as a command followed by a # note, and
    --json gives each as {"command": str or null, "note": str} instead of
    one string mixing the two.

Fixed

  • tether new --shared can join a bookmark whose branch is being written,
    such as a preview environment's database. The plan binds to the branch
    still building on the bookmark's pin, not to its head, which moved on.
  • tether new BOOKMARK (or new REV) to a revision that registers an
    object the current checkout lacks no longer fails with "this new plan
    predates the ref_absent/ref_head ... precondition(s)". Its checks now look
    in the stores the target revision names, which differ when an object was
    moved.
  • Applying a saved new plan refuses to adopt a branch that has a newer
    generation (.2) or is gone. 0.1.0b5's plans bound an adopt to its head
    only, which a newer generation leaves as it was.
  • restore --at and add --at refuse an empty or blank REF. An unset
    variable (--at "$UNSET") used to reset the branch from the upstream head.
    --at 0 (and at=0 from Python) is a version like any other, such as a
    Delta table's first.
  • A file or neon object registered with at = 0 is refused like any
    other at, instead of reading the current head.
  • tether ops shows a restore --at 0 as restored KEY from 0, not from None.
  • tether recover lists per-workspace branches from before bookmarks, which
    may hold uncommitted writes, with a restore --at step for each. It
    suggests no dataset id when run on some keys only, and puts tether repair
    first when a manifest's pin is missing from its store. It also matches
    escaped bookmark names (feature/x) to the VCS's bookmarks.
  • tether recover KEY... no longer suggests a tether commit, which pins
    every object under the current id and could lock in the wrong one; its
    last step is tether recover without keys.
  • tether recover's commands quote every key, ref, bookmark and message for
    the shell, and its advice for a legacy branch that objects share (two Neon
    databases on one branch) names them in one tether restore, which
    restore requires.
  • tether recover KEY... suggests tether repair -- KEY... for only the
    selected objects whose pins are missing, instead of a repair of every
    object that reached the stores the run left out.
  • tether recover's legacy-branch steps run from the trunk: each moves to a
    bookmark recover-<workspace> off the trunk first, since restore --at
    refuses the trunk, and commits what it restored.
  • tether recover's commands end their options with --, so a key such as
    --help or -x, or a git branch name starting with -, is not read as
    an option.
  • tether recover joins an existing recover-<workspace> bookmark only when
    it is an earlier recovery's (nothing but its commits, and the objects
    registered); an unrelated bookmark of that name is left alone and the steps
    use recover-<workspace>-2.
  • git: a branch whose name starts with - (git update-ref makes one) is
    read as a name by every git call tether makes, so tether new -- -feat
    switches to it instead of failing to resolve it.

tether 0.1.0b5

tether 0.1.0b5 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 30 Sep 22:53

Added

  • tether add LOCATOR --kind KIND (and Repo.add(None, ...)) takes the key
    from the locator's last segment, less the store suffix, for file, icechunk,
    lance, delta and git objects. Other kinds still need a key.
  • status, snapshot and verify take KEY..., and diff a repeatable
    --key: an exact key or a prefix ending in / (zarr/). Only those
    objects are contacted and reported. In Python it is keys= on each
    method, and Repo.select_keys, which promote and restore now use too.
    Repo.restore([]) is refused rather than resetting every object;
    keys=None asks for all of them.
  • commit KEY... and pull --key KEY commit only those objects, as git commit PATH does. Other manifests and uncommitted .tether/ edits stay
    out of the commit, and objects sharing a branch space are named together.
    A saved plan records its selection under its digest.

Fixed

  • secrets.toml adds to a table tether.toml also sets, instead of
    replacing it: a committed Iceberg catalog keeps its type and
    warehouse when secrets.toml adds the catalog's uri, and a committed
    storage_options.region survives a secrets file that adds an endpoint. A
    [uris."..."] or [objects."..."] entry adds to the kind's tables the same
    way, and an Iceberg locator's catalog adds to the kind's.
  • file, delta and lance: a region given in more than one place
    (secrets.toml, the locator, storage_options under any spelling) reaches
    the store once, the most specific winning; obstore refused it as a
    duplicate. Icechunk's store checks also honour a locator region.
  • Object keys with spaces, parentheses or [ reach jj and git quoted: jj
    rejected such a path as a revset, and git read [ab] as a pattern that
    could sweep another object's manifest into a commit.

tether 0.1.0b4

tether 0.1.0b4 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 24 Sep 15:16

Security

  • git: a manifest's path must be absolute and outside the checkout; git
    runs no fsmonitor, hook, ext:: transport or implicit bare repository, and
    ignores an inherited GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE or
    GIT_CONFIG_*.
  • git: a committed remote must name a configured remote; a URL goes in
    .tether/secrets.toml ([objects."<key>"] remote).
  • import: [import] query is read from .tether/secrets.toml, not
    tether.toml, and runs as one read-only statement.
  • dolt: credentials come only from the server's [uris."mysql://host:port"]
    entry in .tether/secrets.toml (password_env / user_env move there);
    $DOLT_PASSWORD went to any host a manifest named.
  • Object keys may not contain \ or a drive letter, and are checked when a
    manifest is read.
  • .tether/secrets.toml, workspace.toml and ops.jsonl are refused while
    jj or git tracks them, with the command that untracks them. The committed
    .tether/.gitignore was all that kept them out, so a cloned
    secrets.toml naming git_path ran that program on tether status.
  • git 2.38 is the minimum version, checked by the VCS adapter and the git
    backend; an older git ignores safe.bareRepository without a word.
  • Another live checkout's workspace.toml or ops.jsonl that the VCS tracks
    there is skipped, with a warning naming the checkout: a shipped op log could
    make gc release pins as this clone's.
  • open checks again whether the VCS tracks workspace.toml whenever the
    file has changed, so a long-lived Repo refuses one that a pull or commit
    put under version control after find.

Added

  • ObjectBackend.fork, promote and merge take expected=, the head the
    caller reviewed (or ABSENT): the ref moves only from there, else
    RefMovedError and nothing moves. Backends without the keyword work as
    before; the engine passes the heads its plans reviewed.

  • tether.plan.REQUIRED_PRECONDITIONS per command, and a workspace_bookmark
    precondition: the checkout's bookmark as workspace.toml and the VCS see it.

  • The conformance suite checks conditional forks, PROMOTE, MERGE,
    ancestor_of and opening an older recorded state.

  • gc --release-foreign (Repo.gc(release_foreign=)): also release
    unreferenced pins this clone did not create.

  • icechunk: allow_http and force_path_style in .tether/secrets.toml
    (per URI prefix or object) for an S3-compatible server such as SeaweedFS
    or MinIO; without them add --create could not reach one.

  • Capability.CONDITIONAL_REF declares that a backend's ref moves honour
    expected; conformance fails a backend that claims it and ignores it.

  • new --shared adopts a peer's uncommitted writes when they build on the
    bookmark's pin, instead of asking for --discard.

Changed

  • Partial success (an undo, repair, upgrade or forget-workspace that
    could not do everything) exits 3; 2 is Click's usage error.

  • --help keeps bracketed text such as [experimental]; add --kind lists
    each kind with its maturity.

  • A plan must carry the preconditions its command requires; one saved by an
    older tether, or edited, is refused as stale. commit, new, restore,
    promote, drop, gc and repair plans bind to the checkout that made
    them, and commit, restore, promote and drop plans to its bookmark;
    import, upgrade and forget-workspace plans bind to no checkout.

  • promote lands committed states only (tether commit uncommitted writes
    first). Merges run before fast-forwards, which are held when a merge does
    not land; the trunk moves to the commit the plan reviewed.

  • restore checks the head of every branch it would reset, deferred forks
    included, wants --discard for uncommitted writes, and refuses a head it
    cannot read.

  • undo reverses the newest operation only and refuses one it cannot undo
    rather than reaching past it; tether undo ID restores only the
    workspace.toml fields that entry changed. The new and gc a drop runs
    are its steps ((step of ID) in tether ops) and cannot be undone alone.

  • new, restore and the first writable open hold the repository lock
    while they check and fork, and fork only onto the head the plan saw.

  • Without fcntl (Windows), writing commands are refused; status, verify,
    diff, log, ops and gc --dry-run work.

  • gc and drop release only pins this clone created (recorded in
    tether-pinned.jsonl beside the repository lock, seeded from the op logs).
    Any other unreferenced pin is kept as informational keep-pin until it is
    fetched or --release-foreign is passed; GcReport.kept_pins and gc --json list them.

  • jj 0.43 is the minimum version; an older one is refused.

  • jj and git run with tether's own colour and pager settings, whatever the
    user's config says; tether tracks its own files by name, and its revsets
    use no name an alias can redefine. Colour forced on, all() aliased or
    auto-tracking off had corrupted commit ids, made empty commits, or shrunk
    the history gc walks.

  • file, icechunk, lance, delta: every spelling of a local path --
    /p, /p/, file:///p, a path through a symlinked parent such as macOS's
    /tmp -- is one store to pin ids, listings and gc. New pins of an
    object registered under another spelling get new ids.

  • neon: pins are unprotected unless protected_pins = true; Free has no
    protected branches, and paid plans allow a few.

  • The dataset format is version 5: run tether upgrade once on a 0.1.0b3
    dataset. It gives the stores in tether-touched.jsonl and
    tether-created.jsonl their new identities, stores listings again under
    their new names, records Lance (branch_id), Neon (commit_xid) and
    directory (symlinks) states in the new form where the data is unchanged,
    and makes DuckLake paths absolute. 0.1.0b3 refuses a version 5 dataset,
    so clones on the two releases cannot take turns.

  • Saved plans are format 3, with a digest binding their actions and context
    to their preconditions; re-run a plan saved in format 1 or 2.

  • gc and promote print what they applied along with the failures, and
    exit 3 when part of the work was done.

  • jj calls keep only your identity, signing, snapshot and git settings, and
    your immutable_heads() with the revset aliases it names.

  • A new file of yours that jj has not snapshotted stays in the change it was
    made in when tether moves the working copy.

  • Every open follows the checkout's current bookmark; on Windows a default
    open is read-only.

Removed

  • The lakefs backend, tether add --repository/--prefix, LakeFSHandle
    and the lakefs extra.

Fixed

  • Delta history and diff attached the wrong commit to each version below
    the head.

  • Lance states off main carry the branch id, so a state from a re-created
    branch verifies as missing instead of opening another branch's data.

  • tether diff with no arguments compares against jj's @-, not the working
    copy commit.

  • file: allow_http and the other HTTP client options work on S3; symlinks
    to directories and dangling ones count by their target; the racy-timestamp
    guard covers every timestamp granularity.

  • tether status labels an unreadable object error, reports the rest and
    exits 1 instead of aborting.

  • tether init --json prints only JSON.

  • Two --shared checkouts materializing one lazy fork could throw the first
    one's write away.

  • Undoing an older new after a commit on its branch deleted the branch, for
    pin = "record" the only copy of that state; it now needs --discard, and
    the workspace no longer rolls back to that time's bookmark. Undoing an older
    add moved the checkout to main, so the next write went to the store's
    main.

  • Two undos after a drop revived the abandoned commit.

  • jj: undoing a commit other commits were built on rewrote them; refused now
    (jj backout reverts in place).

  • promote landed uncommitted writes and moved the trunk to a commit
    recording an older state; a conflicted merge left its fast-forwards landed;
    a saved plan applied on another bookmark moved the trunk there; the reset
    after a merge discarded a concurrent write.

  • A long-lived Repo's writable open ignored a new another process ran
    since it was constructed.

  • new kept the previous bookmark's snapshot cache, so status and
    commit --no-snapshot used the old branch's head under the new bookmark.

  • AWS profile or role credentials were never refreshed (now five minutes
    before expiry), and two prefixes of one bucket with different credential
    rules shared the first's.

  • The locks were re-entrant per Repo, not per thread; a second thread could
    leave that Repo unable to lock again.

  • jj: drop from a bookmark whose working copy had edits planned no leave and
    left workspace.toml naming the dropped bookmark.

  • gc counts every live checkout's working-tree manifests and the pins of
    running or interrupted operations as references; --prune-bookmarks keeps
    every branch a live checkout works on or has pending.

  • gc and drop refuse while jj reports a conflicted bookmark or a
    .tether/ conflict no later commit resolved, and promote while its trunk
    is conflicted; status and commit name a conflicted bookmark instead of
    calling it gone.

  • commit raises when the new commit's tree lacks a manifest (a dataset under
    an ignored directory made an empty commit status called clean).

  • git: a hook-refused git commit left the manifests staged; the index is
    reset.

  • jj: the history walk reads every side of a conflicted commit, so gc
    counts the pins each side names.

  • file: a local path holding # or ? was cut short there, and
    add --create on a file:// URI made a stray file: directory.

  • A saved drop plan applies only in the checkout that made it, and only
    while that checkout is still on (or of...

Read more

tether 0.1.0b3

tether 0.1.0b3 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 18 Sep 22:50

Experimental

  • neon: database is out of the object identity, as role already was. A
    Neon branch at an LSN is a snapshot of the whole project, so two objects on
    two databases of one project are one snapshot to tether: one pin branch per
    commit instead of two identical ones cut off the same LSN, one working
    branch per bookmark (as branch_scope already arranged), and open on
    each connects to its own database through them. No migration: existing
    pins keep their recorded ids and refs, stay referenced by the history that
    made them, and verify and repair as before; the next commit of content
    already pinned under an old id pins it once more under the new one.

Fixed

  • commit pins once per pin id. Two objects with one identity and one
    content state name one snapshot; the second is now recorded at the state
    the pin was cut at instead of its own fingerprint of the same content,
    whose volatile address (a Neon LSN) could differ and made the backend
    refuse the second pin as hanging off the wrong LSN.

tether 0.1.0b2

tether 0.1.0b2 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 17 Sep 19:30

Added

  • tether drop BOOKMARK (Repo.plan_drop / apply_drop / drop): throwing
    a bookmark away is one command, the opposite of promote. In order: leave
    the bookmark when this checkout is on it (--to, default the trunk), drop
    the commits only it reaches (jj: jj abandon; git: nothing reaches them
    once the branch is gone), delete it, then the store side -- gc --prune-bookmarks restricted to its branches, the pins nothing references
    once the commits are gone, and with --delete-stores the experimental
    created-store step. One rule is drop's own: a branch whose head a dropped
    commit pinned or recorded is deleted (committed work thrown away by name),
    where gc keeps it as "unpinned writes"; uncommitted writes still need
    --force-prune. Dry-run by default; the plan previews the store side as it
    will be once the commits are gone and apply re-plans it live; the set of
    commits only the bookmark reaches is re-derived at apply and a saved plan is
    refused if another bookmark has come to reach one of them. "Only the
    bookmark reaches" counts every reacher the VCS knows -- other bookmarks,
    tags, remote bookmarks, other workspaces' working copies -- and the plan
    notes a feature@origin that still reaches the line. Refused for the
    trunk, for a bookmark another live checkout works on, and when --to
    names one; not undoable by tether (the CLI guide gives the three-step
    recovery). New plan verbs leave-bookmark, abandon-commit,
    delete-bookmark; precondition bookmark_head; GcScope for
    plan_gc(scope=); VcsAdapter.exclusive_commits / remote_counterparts
    / files_at_many / drop_bookmark. The use-cases story's three
    retirements are one line each now. abandon REV [--gc] stays as the
    surgical form: commits off a bookmark you keep.

Fixed

  • keep-store is an informational plan action: a gc plan holding only
    keep-store lines is empty, like one holding only keep-branch.

tether 0.1.0b1

tether 0.1.0b1 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 16 Sep 04:47

The alpha-exit release, and the first beta. Two security fixes, the engine
bugs an external review found, a hardened backend contract, one migration
for every alpha format (removed at 0.1.0 -- see Deprecated), an
experimental package that is an import boundary with a seamless graduation
path, a narrower undo that reverses only what an operation created,
tether.repo as a package of one module per command family, and -- as an
experimental feature -- a store lifecycle: add --create makes a store
tether owns and gc --delete-stores reclaims it. What still has to run
against real services, and what graduates or goes before 0.1.0, is in
ROADMAP.md.

Upgrade with tether upgrade; move [vcs] git_path/jj_path,
[backends.neon], [backends.ducklake] init_sql, [backends.lakefs], and
any endpoint or credential option out of tether.toml into
.tether/secrets.toml.

Security

  • A cloned dataset is untrusted input. The committed tether.toml could
    choose the executables tether runs ([vcs] git_path/jj_path), the
    endpoint credentials are sent to ([backends.neon] api_url,
    storage_options.endpoint, lakeFS client kwargs), SQL to run
    ([backends.ducklake] init_sql), and which environment variable is sent as
    a password ([backends.dolt] password_env). Backends now declare
    SAFE_CONFIG_KEYS; a committed key outside the allowlist -- or an
    endpoint-/credential-shaped key inside an allowed option table -- is
    refused with a message saying where it belongs. Those settings live in the
    new untracked .tether/secrets.toml ([vcs], [backends.<kind>]) or the
    environment (TETHER_GIT, TETHER_JJ). An Iceberg locator's catalog
    table is screened the same way.
  • git argument injection. Manifest and state values (ref, at,
    remote, a pin's ref, a sha) reached git positionally without
    --end-of-options, so at = "--output=FILE" made git log write FILE.
    Every git call now passes --end-of-options before its positionals, a
    sha must be hex, and a ref/at/remote/path beginning with - is
    refused at add (ObjectBackend.validate_locator) and at use.

Fixed

  • promote moved the trunk bookmark backwards or sideways when the trunk had
    gained commits the bookmark lacked, dropping them off main. A full
    promotion is refused at plan unless the trunk is an ancestor of the
    bookmark's commit (merge or rebase the manifests first, or name keys); at
    apply the trunk is never moved backwards (PromoteReport.trunk_held).
  • new on an existing bookmark (reuse) overwrote the branch's fork point
    with its own head, so the next promote saw a spurious "base moved" and
    merged (or refused) where a fast-forward was right. A kept branch keeps its
    fork point, and promote asks the store's own history first
    (ancestor_of), using the recorded fork point only where the backend has
    no DAG.
  • Neon put the branch name in the content state, so an untouched fork read
    as modified, commit pinned a child of the working branch for no data
    change, and new on the bookmark demanded --discard. The state's
    branch is now the lineage (the object's source branch when the fork
    descends from it) and the timeline actually read is the volatile
    timeline, so a fork with no writes has the pin's content -- the Lance
    pattern. Neon runs the shared conformance suite and a full Repo lifecycle
    against the API fake; the suite's stability and fork checks compare
    content states (up to VOLATILE_KEYS).
  • status spawned two VCS processes per commit in the op log to detect
    drift; vcs_drift now asks once (VcsAdapter.alive_commits).
  • Opening a dataset rewrote .tether/.gitignore; Repo.find now writes it
    only when an untracked file that exists is not yet ignored (so an older
    dataset's new secrets.toml never reaches a commit). init and upgrade
    write the full list.
  • The checkout lock failed outright when another command held it; it now
    waits up to Repo.LOCK_TIMEOUT (30 s) like the repository lock, and the
    no-fcntl branch is re-entrant.
  • A saved gc plan under jj went stale after any snapshot: the digest now
    covers all() ~ working_copies() and binds to the working copy's parent.
  • name.2 working refs (a Neon or Lance sibling of a branch that could not be
    reset) are parsed back to their bookmark, and an 8-hex bookmark with a
    suffix is no longer mistaken for a legacy workspace id; new -b refuses a
    bookmark name ending in .<number>.

Deprecated

  • The alpha upgrade path. tether.upgrade (the one composed migration
    from any 0.1.0aN format, its history rewriters, and the legacy working-ref
    parsing) ships with the 0.1.0 betas and is removed at 0.1.0. After that, a
    dataset at an alpha version fails at open with a message naming the last
    beta: install tether-vcs==<last beta>, run tether upgrade, reinstall.
    Repo.plan_upgrade / apply_upgrade / upgrade are thin delegates that
    import the package on first use; repo.py and vcs.py read without it.
    tether.migrations is now tether.upgrade.migrations; UpgradeReport
    still imports from tether.

Changed

  • tether.repo is a package. The 5,500-line repo.py is split by
    command family -- _core (state, locks, construction, backends, the op
    log, plan verification), _objects (add/remove, set, pull, snapshot and
    status, open, history, verify, diff), _commit, _fork (new, restore),
    _promote, _gc (gc, forget-workspace, abandon), _undo (undo, repair),
    and _reports (the result dataclasses) -- each a mixin over RepoCore,
    assembled into the same public Repo. A pure move: the same 128 methods,
    the same tether.repo exports. Along the way the four workspace-walking
    loops became one _iter_live_workspaces(), and plan_promote reads every
    object's base and working branch in one fan-out instead of two round
    trips per object.
  • Round-2 review corrections: committed option tables (storage_options,
    catalog) are screened by a per-backend SAFE_OPTION_KEYS allowlist
    rather than a substring blocklist, so an option tether has never named is
    refused by name; Repo.backend_for reads the class contract via
    backend_class() instead of building a probe instance; tether open
    prints a Neon connection URL with the password redacted unless
    --with-password (never under --json); Icechunk, Delta, and DuckLake
    implement validate_locator (URI scheme, numeric at); Neon takes one
    branch listing per operation and documents the next_xid collision
    between sibling branches; PromoteReport.trunk_moved has its docstring
    back.
  • Plan preconditions. What a plan saw is recorded as typed
    Plan.preconditions (manifest_hash, workspace_id, vcs_head,
    history_digest, config_version, ref_absent, ref_head, base_state,
    pin_state, no_new_holders) instead of being re-implemented inline per
    command; one Repo._verify_plan() runs them before any apply_* acts.
    Saved plans are format 2; a format-1 plan (before 0.1.0b1) is refused with
    "re-run the plan". Same semantics for every command, one place to audit
    the drift contract.
  • Hygiene: forget-workspace lost its dead delete-branch/keep-branch
    actions and the --force-prune compatibility flag (branches belong to
    bookmarks; gc --prune-bookmarks judges them); Handle.key is documented
    as a display label, not an identity; the docs describe staleness as the
    code enforces it (detected per object, refused workspace-wide) and drop
    the stale --prune-workspaces / --write direct references; the test
    suite finds jj on PATH (or TETHER_TEST_BIN) instead of a hard-coded
    path.
  • undo reverses what an operation created, and reports the rest.
    undo new/fork/restore delete the branches the op created and restore
    workspace.toml and the VCS position; a branch the op reset is no longer
    re-pointed to a recorded head (the store may not allow it and the head to
    choose is yours) -- it is reported with its old head and the tool that
    moves it (restore KEY --from REV, new --discard). undo gc restores
    forgotten working refs and listings but no longer recreates deleted
    branches (repair does, from the manifests). undo --to and
    Repo.undo_to are gone: several slips are several undos, newest first.
  • tether.experimental. The backends tested only against fakes (Neon,
    lakeFS, Dolt, DuckLake, Iceberg) moved to tether.experimental.backends,
    and the registry layer (export, publish, import) to
    tether.experimental.registry. Nothing users type or import changes: kind
    names, extras, CLI commands, the Repo methods, and the
    tether.ExportBundle / ImportSpec / ImportReport / PublishReport /
    build_bundle / specs_from_rows re-exports are the stable surface.
    It is an import boundary: Repo.export/plan_import/apply_import/
    import_objects are thin delegates to tether.experimental.registry.ops
    imported on first call, the tether.<Symbol> names resolve lazily, and
    the CLI commands are attached from tether.experimental.cli, so
    import tether loads none of it. The alpha-era module paths
    (tether.backends.{neon,lakefs,dolt,ducklake,iceberg}, tether.export,
    tether.registry) are removed without a deprecation window.
    export/publish/import print the experimental note add already
    printed for experimental kinds. Graduating a backend is a file move plus
    MATURITY = "stable" (documented in Extending).
  • One migration. tether upgrade brings any alpha dataset to the
    current version in a single step whose parts run on what the dataset shows
    (old-format pins, write = or mtime file states, misplaced manifests or
    relative locators), not on its recorded version. One version write at the
    end, one VCS commit; store renames still fail closed before any history
    rewrite. The plan records its `parts...
Read more

tether 0.1.0a10

tether 0.1.0a10 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 14 Sep 16:15

0.1.0a9 was tagged in history but never published; a10 is the first release
carrying both sets of changes. Datasets created with a8 need tether upgrade
(v3 and v4 migrations; working tree only, no history rewrite).

Added

  • The op log is a journal. Store-writing commands (commit, pull,
    new, the lazy fork, gc, promote, restore, repair) write their
    entry -- plan and what undo needs -- and sync it before the first side
    effect, then a completion mark with the result. An interrupted run leaves an
    INCOMPLETE entry: ops flags it, undo skips it (and says why when
    named), repair --dry-run lists it with what gc will collect
    (Repo.incomplete_ops()).
  • One writer per checkout. Writing commands hold .tether/lock (flock,
    re-entrant per Repo) so two tether processes cannot interleave journal
    entries and workspace.toml writes.
  • Branch scope. ObjectBackend.branch_scope(locator) names the resource
    that owns branches and ref_namespace(locator) the one whose pins
    list_pins returns (both default to the canonical identity; Neon: the
    project, Iceberg: the table). new forks one branch per scope under a
    bookmark -- the first member forks, later members share it, a member that
    pins a different state of the same branch is refused -- and gc collects
    the pins every object references per namespace.
  • Pin.created (runtime-only): whether pin() made the ref or found it
    already carrying the state. The conformance suite checks both answers.
  • DiffEntry.why: which of state, pin, locator, policy differ. A
    locator- or policy-only change is changed (CLI: [policy changed; same state]; --json carries why).
  • tether backends lists kinds with maturity, tier, and capabilities;
    tether --version. Backends declare MATURITY (stable: full lifecycle
    against the real system in CI; experimental: tested against a fake of a
    network service -- neon, lakefs, ducklake, dolt); add notes an
    experimental kind.
  • ObjectBackend.LOCAL_PATH_KEYS: locator keys that may hold a local path.
    The git backend runs the shared conformance suite.
  • ObjectBackend.state_addressable(locator, state): whether a particular
    recorded state can be reopened (the file backend says no for a remote
    object without a version id); commit records such a state
    recoverable = false and says why in the plan.
  • VcsAdapter.history_digest(): a digest of every visible commit id, across
    workspaces and bookmarks; gc plans bind to it.
  • A repository-wide lock. commit, pull, gc, undo, and abandon
    hold tether.lock in the store every checkout shares
    (VcsAdapter.shared_dir(): git's common dir, jj's repo dir), waiting up to
    Repo.REPO_LOCK_TIMEOUT, so a gc in one workspace cannot race a commit in
    another between deciding a pin is unreferenced and releasing it.
  • Progress records. Every side effect of a journaled operation appends a
    record (mark_progress; OpEntry.progress): each pin and the VCS commit of
    a commit, each fork of new/restore, each unpin and deletion of gc,
    each system a promote lands, each repin/refork of repair. repair --dry-run lists what an incomplete operation got done, how many actions
    were planned, and the re-run contract (running the command again finishes
    what is left).

Changed

  • Config v4 (tether upgrade; working tree only, no history rewrite).
    Manifest paths append .toml to the key's last segment instead of
    replacing its suffix, so foo and foo.bar no longer share
    objects/foo.toml (the migration moves misplaced files; a collision that
    already destroyed a manifest is reported). Keys are validated: no empty
    segments, ./.., leading slash, or control characters. Relative local
    paths in locators are resolved against the dataset root (the migration
    rewrites them); add and import resolve a relative path against the
    caller's directory and store it absolute.
  • A pin is verified before it is read or forked. open --rev, new from
    a commit, and promote --rev check the pin against the manifest's state:
    a deleted pin falls back to the recorded state where the backend can
    address it; a moved pin raises PinDriftError (a refuse in a promote
    plan). repair never overwrites a drifted pin.
  • Destructive steps re-check the ref they act on. Plans record the head
    of every branch they delete or reset; apply reads it again immediately
    before the step and stops with StalePlanError when it moved: gc
    delete-branch (gc plans are also bound to the VCS head and manifest
    hash), new's reuse/reset (plus a re-run of the bookmark-holder guard and
    a same-workspace check), restore's reset, promote's source, repair's
    refork (the branch must still be missing). A lazy fork resets an existing
    branch only onto the head new reviewed (workspace.toml
    pending_resets), reuses a branch already at the pin or one a scope
    sibling writes through, and otherwise refuses. gc plans are bound to the
    digest of all visible history (a commit made in another workspace can
    reference a pin the plan would release) and check every branch head before
    the first action, so a stale plan does nothing at all; promote --rev
    verifies a pin source still names the reviewed state; a fork new planned
    as fresh is re-checked for absence at apply, and new refuses outright
    when the backend cannot list branches (unknown is not absent).
  • commit compensates as a unit. A failure after the pins -- manifest
    write, listing, VCS commit -- releases only the pins this commit created
    (never a reused one), restores the manifests and listings it wrote, and
    journals the attempt as failed and rolled back. If the VCS commit landed
    before the adapter raised, nothing is rolled back: history names the pins,
    so the operation completes as a commit that succeeded and the trailing
    error is surfaced (failed_after_commit).
  • restore and promote are closed over the branch scope. Restoring one
    of several keys that write through one branch is refused (the message names
    the siblings to include); with all named, the branch is reset once and the
    siblings share it. promote fast-forwards a shared branch once instead of
    once per key.
  • The writer lock also covers add, remove, set, import, abandon,
    forget-workspace, and upgrade; undo journals before it acts (a refused
    undo is recorded as failed). Taking the lock re-reads workspace.toml and
    the manifests, so a long-lived Repo never writes the state it loaded at
    construction over what another process wrote since; snapshot writes its
    cache under the lock.
  • undo completes atomically: the undone mark rides in the done record (one
    append), and a handler refusal that touched nothing ends the entry as a
    failed attempt rather than leaving it started.
  • promote KEY... refuses a subset that leaves unnamed siblings whose base
    branch the write would move, whatever the source (a working ref, or a pin
    by --rev), as restore does.
  • promote fast-forwards and merges from the state the plan reviewed, not
    the source ref's current head: what lands is what was shown, whatever the
    timing. ObjectBackend.merge takes str | Pin | State like promote
    (git, lakeFS, Dolt, and memory resolve a state to its commit). The inline
    convenience methods (commit, new, promote, restore, gc, import)
    plan and apply under one checkout lock, and new/promote/restore/
    import re-verify at apply (commit does not need to: a pin names the
    captured state). The CLI's immediate tether commit goes through
    Repo.commit too; only --dry-run/--plan build a separate plan.
  • gc plans take the history digest before walking history, so a commit
    landing during the walk stales the plan instead of slipping between the
    references and the digest.
  • restore checks every head before the first reset and writes the
    workspace after each one (with the siblings sharing the branch), so a kill
    between two resets leaves each reset branch described as such.
  • apply_commit turns a planned key that is no longer registered into
    StalePlanError (rolling back the pins it made before reaching it) rather
    than a KeyError.
  • promote --rev's scope closure and base-head check work from the kind and
    locator the plan captured, so an object removed from the working tree since
    the revision still refuses an unnamed sibling and a base that moved.
  • snapshot and pull run whole under the checkout lock: which refs to read
    is decided from the workspace as it is on disk, not from the one a
    long-lived Repo loaded.
  • new writes workspace.toml -- bookmark set, every fork pending with the
    reset it agreed to -- right after moving the VCS and before the first store
    write, so a process killed in the fork fan-out leaves exactly a lazy new;
    a fork that fails in the fan-out stays pending instead of being forgotten.
  • gc: a branch that moved after the preflight (a race, not a stale plan)
    is kept and reported while the rest of the plan finishes; a --force-prune
    plan that could not read a head refuses at apply if the head reads now.
  • Pre-v4 manifests read from history resolve relative local paths against
    the dataset root, the rule the migration applies to the working tree.
  • promote's guarantee is stated as it is: a bookmark is planned whole or
    not at all; once applying, each system's fast-forward stands on its own.
  • set --pin record on a pinned object takes effect at the next commit (the
    pin is dropped; gc releases the tag once no commit names it); --pin native creates one again. Before, the "unchanged" shortcut kept the pin.
  • file: --file versioned makes only a single remote object Addressable;
    a recorded object state without a version id (unversioned bucket) is
    refused by...
Read more

tether 0.1.0a8

tether 0.1.0a8 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 09 Sep 23:00

Dataset namespaces: pins are tether.. and working branches tether.ws.....; gc never touches another dataset sharing a store. Operation log (.tether/ops.jsonl, per workspace) with tether ops, tether undo [ID | --to ID], and tether repair. tether upgrade brings a1..a7 datasets forward (renames refs in every store and rewrites history to match; fails closed). New entrypoints where the VCS half and the store half must agree: abandon (drops commits + the gc plan they free), restore KEY --from REV (per-object re-fork), forget-workspace. new --discard is required to reset a branch with uncommitted writes; a branch already at the pin is reused (no more Neon sibling per commit). status and ops flag dataset commits removed behind tether's back. Fixes: git pins fail when the remote push fails; Neon role out of identity; Neon API errors are BackendErrors; undeletable branches are planned as kept. BREAKING: [tether] version = 2 -- run tether upgrade on existing datasets.

tether 0.1.0a7

tether 0.1.0a7 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 08 Sep 19:37

Fixes from the second review. States separate content from address (VOLATILE_KEYS / content_state): Iceberg metadata rewrites and Neon checkpoints no longer show as drift or mint duplicate pins; git dirty applies only to the checked-out ref. fork() onto an existing name resets it on every backend, Neon included (restore by head, sibling branch once pins hang off it). Stale detection is per object from recorded base states. Working-ref names carry a key digest and pin ids are 16 hex; earlier alphas' pins and branches are not recognised -- re-commit and new. apply_new verifies before moving the working copy and records successful forks before raising; git new keeps commits on a tether/ branch instead of a detached HEAD; gc --prune-workspaces discovers live jj workspaces and git worktrees. import locator changes drop the old working branch. export/publish/import and lakeFS/Dolt/DuckLake are labelled experimental.

tether 0.1.0a6

tether 0.1.0a6 Pre-release
Pre-release

Choose a tag to compare

@elyall elyall released this 08 Sep 06:12

Lazy forking: tether new decides working branches and the first writable open creates them (--eager / [new] fork = "eager" for the old behaviour; --pin record objects still fork during new). tether promote: fast-forward each system's base branch to the fork, native three-way merge where the system has one (git, lakeFS, Dolt), refuse with a recipe otherwise; fork points recorded in workspace.toml (export schema_version 2). See CHANGELOG.md.