Repository navigation
Releases: Rosebud-Biosciences/tether
Release list
tether 0.1.0b6
Added
tether init --dataset-id IDuses that id instead of a fresh one; anything
but 8 lowercase hex characters is refused.tether restore KEY... --at REFstarts the bookmark's branch as a copy of
a native branch, tag or state id.REFitself is never moved.tether new --adopttakes the bookmark's existing store branches as they
are, uncommitted writes included, instead of resetting them. A saved plan
binds to each branch still existing and being the newest generation, not
to its head.tether recoverlists tether's refs in each store by dataset id and prints
the steps that take back a dataset whose repository was lost.tether repair KEY...(Repo.repair(keys)) rebuilds only those objects'
pins and branches, selected from every key history has had with
--all-history. A saved plan records its selection under its digest.
Changed
- A saved plan (
--plan FILE) records the tether version that made it, and
--from-planapplies it only under that version. Plans saved by any other
version, including every earlier release, are refused: re-run the plan. tether recoverprints each step as a command followed by a#note, and
--jsongives each as{"command": str or null, "note": str}instead of
one string mixing the two.
Fixed
tether new --sharedcan join a bookmark whose branch is being written,
such as a preview environment's database. The plan binds to the branch
still building on the bookmark's pin, not to its head, which moved on.tether new BOOKMARK(ornew REV) to a revision that registers an
object the current checkout lacks no longer fails with "this new plan
predates the ref_absent/ref_head ... precondition(s)". Its checks now look
in the stores the target revision names, which differ when an object was
moved.- Applying a saved
newplan refuses to adopt a branch that has a newer
generation (.2) or is gone. 0.1.0b5's plans bound an adopt to its head
only, which a newer generation leaves as it was. restore --atandadd --atrefuse an empty or blankREF. An unset
variable (--at "$UNSET") used to reset the branch from the upstream head.
--at 0(andat=0from Python) is a version like any other, such as a
Delta table's first.- A
fileorneonobject registered withat = 0is refused like any
otherat, instead of reading the current head. tether opsshows arestore --at 0asrestored KEY from 0, notfrom None.tether recoverlists per-workspace branches from before bookmarks, which
may hold uncommitted writes, with arestore --atstep for each. It
suggests no dataset id when run on some keys only, and putstether repair
first when a manifest's pin is missing from its store. It also matches
escaped bookmark names (feature/x) to the VCS's bookmarks.tether recover KEY...no longer suggests atether commit, which pins
every object under the current id and could lock in the wrong one; its
last step istether recoverwithout keys.tether recover's commands quote every key, ref, bookmark and message for
the shell, and its advice for a legacy branch that objects share (two Neon
databases on one branch) names them in onetether restore, which
restorerequires.tether recover KEY...suggeststether repair -- KEY...for only the
selected objects whose pins are missing, instead of a repair of every
object that reached the stores the run left out.tether recover's legacy-branch steps run from the trunk: each moves to a
bookmarkrecover-<workspace>off the trunk first, sincerestore --at
refuses the trunk, and commits what it restored.tether recover's commands end their options with--, so a key such as
--helpor-x, or a git branch name starting with-, is not read as
an option.tether recoverjoins an existingrecover-<workspace>bookmark only when
it is an earlier recovery's (nothing but its commits, and the objects
registered); an unrelated bookmark of that name is left alone and the steps
userecover-<workspace>-2.- git: a branch whose name starts with
-(git update-refmakes one) is
read as a name by every git call tether makes, sotether new -- -feat
switches to it instead of failing to resolve it.
tether 0.1.0b5
Added
tether add LOCATOR --kind KIND(andRepo.add(None, ...)) takes the key
from the locator's last segment, less the store suffix, for file, icechunk,
lance, delta and git objects. Other kinds still need a key.status,snapshotandverifytakeKEY..., anddiffa repeatable
--key: an exact key or a prefix ending in/(zarr/). Only those
objects are contacted and reported. In Python it iskeys=on each
method, andRepo.select_keys, whichpromoteandrestorenow use too.
Repo.restore([])is refused rather than resetting every object;
keys=Noneasks for all of them.commit KEY...andpull --key KEYcommit only those objects, asgit commit PATHdoes. Other manifests and uncommitted.tether/edits stay
out of the commit, and objects sharing a branch space are named together.
A saved plan records its selection under its digest.
Fixed
secrets.tomladds to a tabletether.tomlalso sets, instead of
replacing it: a committed Icebergcatalogkeeps itstypeand
warehousewhensecrets.tomladds the catalog'suri, and a committed
storage_options.regionsurvives a secrets file that adds an endpoint. A
[uris."..."]or[objects."..."]entry adds to the kind's tables the same
way, and an Iceberg locator'scatalogadds to the kind's.file,deltaandlance: a region given in more than one place
(secrets.toml, the locator,storage_optionsunder any spelling) reaches
the store once, the most specific winning; obstore refused it as a
duplicate. Icechunk's store checks also honour a locator region.- Object keys with spaces, parentheses or
[reach jj and git quoted: jj
rejected such a path as a revset, and git read[ab]as a pattern that
could sweep another object's manifest into a commit.
tether 0.1.0b4
Security
git: a manifest'spathmust be absolute and outside the checkout; git
runs no fsmonitor, hook,ext::transport or implicit bare repository, and
ignores an inheritedGIT_DIR,GIT_WORK_TREE,GIT_INDEX_FILEor
GIT_CONFIG_*.git: a committedremotemust name a configured remote; a URL goes in
.tether/secrets.toml([objects."<key>"] remote).import:[import] queryis read from.tether/secrets.toml, not
tether.toml, and runs as one read-only statement.dolt: credentials come only from the server's[uris."mysql://host:port"]
entry in.tether/secrets.toml(password_env/user_envmove there);
$DOLT_PASSWORDwent to any host a manifest named.- Object keys may not contain
\or a drive letter, and are checked when a
manifest is read. .tether/secrets.toml,workspace.tomlandops.jsonlare refused while
jj or git tracks them, with the command that untracks them. The committed
.tether/.gitignorewas all that kept them out, so a cloned
secrets.tomlnaminggit_pathran that program ontether status.- git 2.38 is the minimum version, checked by the VCS adapter and the
git
backend; an older git ignoressafe.bareRepositorywithout a word. - Another live checkout's
workspace.tomlorops.jsonlthat the VCS tracks
there is skipped, with a warning naming the checkout: a shipped op log could
makegcrelease pins as this clone's. openchecks again whether the VCS tracksworkspace.tomlwhenever the
file has changed, so a long-livedReporefuses one that a pull or commit
put under version control afterfind.
Added
-
ObjectBackend.fork,promoteandmergetakeexpected=, the head the
caller reviewed (orABSENT): the ref moves only from there, else
RefMovedErrorand nothing moves. Backends without the keyword work as
before; the engine passes the heads its plans reviewed. -
tether.plan.REQUIRED_PRECONDITIONSper command, and aworkspace_bookmark
precondition: the checkout's bookmark asworkspace.tomland the VCS see it. -
The conformance suite checks conditional forks,
PROMOTE,MERGE,
ancestor_ofand opening an older recorded state. -
gc --release-foreign(Repo.gc(release_foreign=)): also release
unreferenced pins this clone did not create. -
icechunk:allow_httpandforce_path_stylein.tether/secrets.toml
(per URI prefix or object) for an S3-compatible server such as SeaweedFS
or MinIO; without themadd --createcould not reach one. -
Capability.CONDITIONAL_REFdeclares that a backend's ref moves honour
expected; conformance fails a backend that claims it and ignores it. -
new --sharedadopts a peer's uncommitted writes when they build on the
bookmark's pin, instead of asking for--discard.
Changed
-
Partial success (an
undo,repair,upgradeorforget-workspacethat
could not do everything) exits 3; 2 is Click's usage error. -
--helpkeeps bracketed text such as[experimental];add --kindlists
each kind with its maturity. -
A plan must carry the preconditions its command requires; one saved by an
older tether, or edited, is refused as stale.commit,new,restore,
promote,drop,gcandrepairplans bind to the checkout that made
them, andcommit,restore,promoteanddropplans to its bookmark;
import,upgradeandforget-workspaceplans bind to no checkout. -
promotelands committed states only (tether commituncommitted writes
first). Merges run before fast-forwards, which are held when a merge does
not land; the trunk moves to the commit the plan reviewed. -
restorechecks the head of every branch it would reset, deferred forks
included, wants--discardfor uncommitted writes, and refuses a head it
cannot read. -
undoreverses the newest operation only and refuses one it cannot undo
rather than reaching past it;tether undo IDrestores only the
workspace.tomlfields that entry changed. Thenewandgcadropruns
are its steps ((step of ID)intether ops) and cannot be undone alone. -
new,restoreand the first writableopenhold the repository lock
while they check and fork, and fork only onto the head the plan saw. -
Without
fcntl(Windows), writing commands are refused;status,verify,
diff,log,opsandgc --dry-runwork. -
gcanddroprelease only pins this clone created (recorded in
tether-pinned.jsonlbeside the repository lock, seeded from the op logs).
Any other unreferenced pin is kept as informationalkeep-pinuntil it is
fetched or--release-foreignis passed;GcReport.kept_pinsandgc --jsonlist them. -
jj 0.43 is the minimum version; an older one is refused.
-
jj and git run with tether's own colour and pager settings, whatever the
user's config says; tether tracks its own files by name, and its revsets
use no name an alias can redefine. Colour forced on,all()aliased or
auto-tracking off had corrupted commit ids, made empty commits, or shrunk
the historygcwalks. -
file,icechunk,lance,delta: every spelling of a local path --
/p,/p/,file:///p, a path through a symlinked parent such as macOS's
/tmp-- is one store to pin ids, listings andgc. New pins of an
object registered under another spelling get new ids. -
neon: pins are unprotected unlessprotected_pins = true; Free has no
protected branches, and paid plans allow a few. -
The dataset format is version 5: run
tether upgradeonce on a 0.1.0b3
dataset. It gives the stores intether-touched.jsonland
tether-created.jsonltheir new identities, stores listings again under
their new names, records Lance (branch_id), Neon (commit_xid) and
directory (symlinks) states in the new form where the data is unchanged,
and makes DuckLake paths absolute. 0.1.0b3 refuses a version 5 dataset,
so clones on the two releases cannot take turns. -
Saved plans are format 3, with a digest binding their actions and context
to their preconditions; re-run a plan saved in format 1 or 2. -
gcandpromoteprint what they applied along with the failures, and
exit 3 when part of the work was done. -
jj calls keep only your identity, signing, snapshot and git settings, and
yourimmutable_heads()with the revset aliases it names. -
A new file of yours that jj has not snapshotted stays in the change it was
made in when tether moves the working copy. -
Every
openfollows the checkout's current bookmark; on Windows a default
openis read-only.
Removed
- The
lakefsbackend,tether add --repository/--prefix,LakeFSHandle
and thelakefsextra.
Fixed
-
Delta
historyanddiffattached the wrong commit to each version below
the head. -
Lance states off
maincarry the branch id, so a state from a re-created
branch verifies as missing instead of opening another branch's data. -
tether diffwith no arguments compares against jj's@-, not the working
copy commit. -
file:allow_httpand the other HTTP client options work on S3; symlinks
to directories and dangling ones count by their target; the racy-timestamp
guard covers every timestamp granularity. -
tether statuslabels an unreadable objecterror, reports the rest and
exits 1 instead of aborting. -
tether init --jsonprints only JSON. -
Two
--sharedcheckouts materializing one lazy fork could throw the first
one's write away. -
Undoing an older
newafter a commit on its branch deleted the branch, for
pin = "record"the only copy of that state; it now needs--discard, and
the workspace no longer rolls back to that time's bookmark. Undoing an older
addmoved the checkout tomain, so the next write went to the store's
main. -
Two
undos after adroprevived the abandoned commit. -
jj: undoing a commit other commits were built on rewrote them; refused now
(jj backoutreverts in place). -
promotelanded uncommitted writes and moved the trunk to a commit
recording an older state; a conflicted merge left its fast-forwards landed;
a saved plan applied on another bookmark moved the trunk there; the reset
after a merge discarded a concurrent write. -
A long-lived
Repo's writableopenignored anewanother process ran
since it was constructed. -
newkept the previous bookmark's snapshot cache, sostatusand
commit --no-snapshotused the old branch's head under the new bookmark. -
AWS profile or role credentials were never refreshed (now five minutes
before expiry), and two prefixes of one bucket with different credential
rules shared the first's. -
The locks were re-entrant per
Repo, not per thread; a second thread could
leave thatRepounable to lock again. -
jj:
dropfrom a bookmark whose working copy had edits planned no leave and
leftworkspace.tomlnaming the dropped bookmark. -
gccounts every live checkout's working-tree manifests and the pins of
running or interrupted operations as references;--prune-bookmarkskeeps
every branch a live checkout works on or has pending. -
gcanddroprefuse while jj reports a conflicted bookmark or a
.tether/conflict no later commit resolved, andpromotewhile its trunk
is conflicted;statusandcommitname a conflicted bookmark instead of
calling it gone. -
commitraises when the new commit's tree lacks a manifest (a dataset under
an ignored directory made an empty commitstatuscalled clean). -
git: a hook-refused
git commitleft the manifests staged; the index is
reset. -
jj: the history walk reads every side of a conflicted commit, so
gc
counts the pins each side names. -
file: a local path holding#or?was cut short there, and
add --createon afile://URI made a strayfile:directory. -
A saved
dropplan applies only in the checkout that made it, and only
while that checkout is still on (or of...
tether 0.1.0b3
Experimental
neon:databaseis out of the object identity, asrolealready was. A
Neon branch at an LSN is a snapshot of the whole project, so two objects on
two databases of one project are one snapshot to tether: one pin branch per
commit instead of two identical ones cut off the same LSN, one working
branch per bookmark (asbranch_scopealready arranged), andopenon
each connects to its own database through them. No migration: existing
pins keep their recorded ids and refs, stay referenced by the history that
made them, and verify and repair as before; the nextcommitof content
already pinned under an old id pins it once more under the new one.
Fixed
commitpins once per pin id. Two objects with one identity and one
content state name one snapshot; the second is now recorded at the state
the pin was cut at instead of its own fingerprint of the same content,
whose volatile address (a Neon LSN) could differ and made the backend
refuse the second pin as hanging off the wrong LSN.
tether 0.1.0b2
Added
tether drop BOOKMARK(Repo.plan_drop/apply_drop/drop): throwing
a bookmark away is one command, the opposite ofpromote. In order: leave
the bookmark when this checkout is on it (--to, default the trunk), drop
the commits only it reaches (jj:jj abandon; git: nothing reaches them
once the branch is gone), delete it, then the store side --gc --prune-bookmarksrestricted to its branches, the pins nothing references
once the commits are gone, and with--delete-storesthe experimental
created-store step. One rule isdrop's own: a branch whose head a dropped
commit pinned or recorded is deleted (committed work thrown away by name),
wheregckeeps it as "unpinned writes"; uncommitted writes still need
--force-prune. Dry-run by default; the plan previews the store side as it
will be once the commits are gone and apply re-plans it live; the set of
commits only the bookmark reaches is re-derived at apply and a saved plan is
refused if another bookmark has come to reach one of them. "Only the
bookmark reaches" counts every reacher the VCS knows -- other bookmarks,
tags, remote bookmarks, other workspaces' working copies -- and the plan
notes afeature@originthat still reaches the line. Refused for the
trunk, for a bookmark another live checkout works on, and when--to
names one; not undoable by tether (the CLI guide gives the three-step
recovery). New plan verbsleave-bookmark,abandon-commit,
delete-bookmark; preconditionbookmark_head;GcScopefor
plan_gc(scope=);VcsAdapter.exclusive_commits/remote_counterparts
/files_at_many/drop_bookmark. The use-cases story's three
retirements are one line each now.abandon REV [--gc]stays as the
surgical form: commits off a bookmark you keep.
Fixed
keep-storeis an informational plan action: a gc plan holding only
keep-storelines is empty, like one holding onlykeep-branch.
tether 0.1.0b1
The alpha-exit release, and the first beta. Two security fixes, the engine
bugs an external review found, a hardened backend contract, one migration
for every alpha format (removed at 0.1.0 -- see Deprecated), an
experimental package that is an import boundary with a seamless graduation
path, a narrower undo that reverses only what an operation created,
tether.repo as a package of one module per command family, and -- as an
experimental feature -- a store lifecycle: add --create makes a store
tether owns and gc --delete-stores reclaims it. What still has to run
against real services, and what graduates or goes before 0.1.0, is in
ROADMAP.md.
Upgrade with tether upgrade; move [vcs] git_path/jj_path,
[backends.neon], [backends.ducklake] init_sql, [backends.lakefs], and
any endpoint or credential option out of tether.toml into
.tether/secrets.toml.
Security
- A cloned dataset is untrusted input. The committed
tether.tomlcould
choose the executables tether runs ([vcs] git_path/jj_path), the
endpoint credentials are sent to ([backends.neon] api_url,
storage_options.endpoint, lakeFS client kwargs), SQL to run
([backends.ducklake] init_sql), and which environment variable is sent as
a password ([backends.dolt] password_env). Backends now declare
SAFE_CONFIG_KEYS; a committed key outside the allowlist -- or an
endpoint-/credential-shaped key inside an allowed option table -- is
refused with a message saying where it belongs. Those settings live in the
new untracked.tether/secrets.toml([vcs],[backends.<kind>]) or the
environment (TETHER_GIT,TETHER_JJ). An Iceberg locator'scatalog
table is screened the same way. - git argument injection. Manifest and state values (
ref,at,
remote, a pin's ref, asha) reached git positionally without
--end-of-options, soat = "--output=FILE"madegit logwrite FILE.
Every git call now passes--end-of-optionsbefore its positionals, a
shamust be hex, and aref/at/remote/pathbeginning with-is
refused atadd(ObjectBackend.validate_locator) and at use.
Fixed
promotemoved the trunk bookmark backwards or sideways when the trunk had
gained commits the bookmark lacked, dropping them offmain. A full
promotion is refused at plan unless the trunk is an ancestor of the
bookmark's commit (merge or rebase the manifests first, or name keys); at
apply the trunk is never moved backwards (PromoteReport.trunk_held).newon an existing bookmark (reuse) overwrote the branch's fork point
with its own head, so the nextpromotesaw a spurious "base moved" and
merged (or refused) where a fast-forward was right. A kept branch keeps its
fork point, andpromoteasks the store's own history first
(ancestor_of), using the recorded fork point only where the backend has
no DAG.- Neon put the branch name in the content state, so an untouched fork read
as modified,commitpinned a child of the working branch for no data
change, andnewon the bookmark demanded--discard. The state's
branchis now the lineage (the object's source branch when the fork
descends from it) and the timeline actually read is the volatile
timeline, so a fork with no writes has the pin's content -- the Lance
pattern. Neon runs the shared conformance suite and a fullRepolifecycle
against the API fake; the suite's stability and fork checks compare
content states (up toVOLATILE_KEYS). statusspawned two VCS processes per commit in the op log to detect
drift;vcs_driftnow asks once (VcsAdapter.alive_commits).- Opening a dataset rewrote
.tether/.gitignore;Repo.findnow writes it
only when an untracked file that exists is not yet ignored (so an older
dataset's newsecrets.tomlnever reaches a commit).initandupgrade
write the full list. - The checkout lock failed outright when another command held it; it now
waits up toRepo.LOCK_TIMEOUT(30 s) like the repository lock, and the
no-fcntlbranch is re-entrant. - A saved
gcplan under jj went stale after any snapshot: the digest now
coversall() ~ working_copies()and binds to the working copy's parent. name.2working refs (a Neon or Lance sibling of a branch that could not be
reset) are parsed back to their bookmark, and an 8-hex bookmark with a
suffix is no longer mistaken for a legacy workspace id;new -brefuses a
bookmark name ending in.<number>.
Deprecated
- The alpha upgrade path.
tether.upgrade(the one composed migration
from any 0.1.0aN format, its history rewriters, and the legacy working-ref
parsing) ships with the 0.1.0 betas and is removed at 0.1.0. After that, a
dataset at an alpha version fails at open with a message naming the last
beta: installtether-vcs==<last beta>, runtether upgrade, reinstall.
Repo.plan_upgrade/apply_upgrade/upgradeare thin delegates that
import the package on first use;repo.pyandvcs.pyread without it.
tether.migrationsis nowtether.upgrade.migrations;UpgradeReport
still imports fromtether.
Changed
tether.repois a package. The 5,500-linerepo.pyis split by
command family --_core(state, locks, construction, backends, the op
log, plan verification),_objects(add/remove, set, pull, snapshot and
status, open, history, verify, diff),_commit,_fork(new, restore),
_promote,_gc(gc, forget-workspace, abandon),_undo(undo, repair),
and_reports(the result dataclasses) -- each a mixin overRepoCore,
assembled into the same publicRepo. A pure move: the same 128 methods,
the sametether.repoexports. Along the way the four workspace-walking
loops became one_iter_live_workspaces(), andplan_promotereads every
object's base and working branch in one fan-out instead of two round
trips per object.- Round-2 review corrections: committed option tables (
storage_options,
catalog) are screened by a per-backendSAFE_OPTION_KEYSallowlist
rather than a substring blocklist, so an option tether has never named is
refused by name;Repo.backend_forreads the class contract via
backend_class()instead of building a probe instance;tether open
prints a Neon connection URL with the password redacted unless
--with-password(never under--json); Icechunk, Delta, and DuckLake
implementvalidate_locator(URI scheme, numericat); Neon takes one
branch listing per operation and documents thenext_xidcollision
between sibling branches;PromoteReport.trunk_movedhas its docstring
back. - Plan preconditions. What a plan saw is recorded as typed
Plan.preconditions(manifest_hash,workspace_id,vcs_head,
history_digest,config_version,ref_absent,ref_head,base_state,
pin_state,no_new_holders) instead of being re-implemented inline per
command; oneRepo._verify_plan()runs them before anyapply_*acts.
Saved plans are format 2; a format-1 plan (before 0.1.0b1) is refused with
"re-run the plan". Same semantics for every command, one place to audit
the drift contract. - Hygiene:
forget-workspacelost its deaddelete-branch/keep-branch
actions and the--force-prunecompatibility flag (branches belong to
bookmarks;gc --prune-bookmarksjudges them);Handle.keyis documented
as a display label, not an identity; the docs describe staleness as the
code enforces it (detected per object, refused workspace-wide) and drop
the stale--prune-workspaces/--write directreferences; the test
suite findsjjonPATH(orTETHER_TEST_BIN) instead of a hard-coded
path. undoreverses what an operation created, and reports the rest.
undo new/fork/restoredelete the branches the op created and restore
workspace.tomland the VCS position; a branch the op reset is no longer
re-pointed to a recorded head (the store may not allow it and the head to
choose is yours) -- it is reported with its old head and the tool that
moves it (restore KEY --from REV,new --discard).undo gcrestores
forgotten working refs and listings but no longer recreates deleted
branches (repairdoes, from the manifests).undo --toand
Repo.undo_toare gone: several slips are severalundos, newest first.tether.experimental. The backends tested only against fakes (Neon,
lakeFS, Dolt, DuckLake, Iceberg) moved totether.experimental.backends,
and the registry layer (export,publish,import) to
tether.experimental.registry. Nothing users type or import changes: kind
names, extras, CLI commands, theRepomethods, and the
tether.ExportBundle/ImportSpec/ImportReport/PublishReport/
build_bundle/specs_from_rowsre-exports are the stable surface.
It is an import boundary:Repo.export/plan_import/apply_import/
import_objectsare thin delegates totether.experimental.registry.ops
imported on first call, thetether.<Symbol>names resolve lazily, and
the CLI commands are attached fromtether.experimental.cli, so
import tetherloads none of it. The alpha-era module paths
(tether.backends.{neon,lakefs,dolt,ducklake,iceberg},tether.export,
tether.registry) are removed without a deprecation window.
export/publish/importprint the experimental noteaddalready
printed for experimental kinds. Graduating a backend is a file move plus
MATURITY = "stable"(documented in Extending).- One migration.
tether upgradebrings any alpha dataset to the
current version in a single step whose parts run on what the dataset shows
(old-format pins,write =or mtime file states, misplaced manifests or
relative locators), not on its recorded version. One version write at the
end, one VCS commit; store renames still fail closed before any history
rewrite. The plan records its `parts...
tether 0.1.0a10
0.1.0a9 was tagged in history but never published; a10 is the first release
carrying both sets of changes. Datasets created with a8 need tether upgrade
(v3 and v4 migrations; working tree only, no history rewrite).
Added
- The op log is a journal. Store-writing commands (
commit,pull,
new, the lazy fork,gc,promote,restore,repair) write their
entry -- plan and whatundoneeds -- and sync it before the first side
effect, then a completion mark with the result. An interrupted run leaves an
INCOMPLETEentry:opsflags it,undoskips it (and says why when
named),repair --dry-runlists it with whatgcwill collect
(Repo.incomplete_ops()). - One writer per checkout. Writing commands hold
.tether/lock(flock,
re-entrant perRepo) so twotetherprocesses cannot interleave journal
entries andworkspace.tomlwrites. - Branch scope.
ObjectBackend.branch_scope(locator)names the resource
that owns branches andref_namespace(locator)the one whose pins
list_pinsreturns (both default to the canonical identity; Neon: the
project, Iceberg: the table).newforks one branch per scope under a
bookmark -- the first member forks, later membersshareit, a member that
pins a different state of the same branch is refused -- andgccollects
the pins every object references per namespace. Pin.created(runtime-only): whetherpin()made the ref or found it
already carrying the state. The conformance suite checks both answers.DiffEntry.why: which ofstate,pin,locator,policydiffer. A
locator- or policy-only change ischanged(CLI:[policy changed; same state];--jsoncarrieswhy).tether backendslists kinds with maturity, tier, and capabilities;
tether --version. Backends declareMATURITY(stable: full lifecycle
against the real system in CI;experimental: tested against a fake of a
network service -- neon, lakefs, ducklake, dolt);addnotes an
experimental kind.ObjectBackend.LOCAL_PATH_KEYS: locator keys that may hold a local path.
The git backend runs the shared conformance suite.ObjectBackend.state_addressable(locator, state): whether a particular
recorded state can be reopened (thefilebackend says no for a remote
object without a version id);commitrecords such a state
recoverable = falseand says why in the plan.VcsAdapter.history_digest(): a digest of every visible commit id, across
workspaces and bookmarks;gcplans bind to it.- A repository-wide lock.
commit,pull,gc,undo, andabandon
holdtether.lockin the store every checkout shares
(VcsAdapter.shared_dir(): git's common dir, jj's repo dir), waiting up to
Repo.REPO_LOCK_TIMEOUT, so a gc in one workspace cannot race a commit in
another between deciding a pin is unreferenced and releasing it. - Progress records. Every side effect of a journaled operation appends a
record (mark_progress;OpEntry.progress): each pin and the VCS commit of
acommit, each fork ofnew/restore, each unpin and deletion ofgc,
each system apromotelands, each repin/refork ofrepair.repair --dry-runlists what an incomplete operation got done, how many actions
were planned, and the re-run contract (running the command again finishes
what is left).
Changed
- Config v4 (
tether upgrade; working tree only, no history rewrite).
Manifest paths append.tomlto the key's last segment instead of
replacing its suffix, sofooandfoo.barno longer share
objects/foo.toml(the migration moves misplaced files; a collision that
already destroyed a manifest is reported). Keys are validated: no empty
segments,./.., leading slash, or control characters. Relative local
paths in locators are resolved against the dataset root (the migration
rewrites them);addandimportresolve a relative path against the
caller's directory and store it absolute. - A pin is verified before it is read or forked.
open --rev,newfrom
a commit, andpromote --revcheck the pin against the manifest's state:
a deleted pin falls back to the recorded state where the backend can
address it; a moved pin raisesPinDriftError(arefusein a promote
plan).repairnever overwrites a drifted pin. - Destructive steps re-check the ref they act on. Plans record the head
of every branch they delete or reset; apply reads it again immediately
before the step and stops withStalePlanErrorwhen it moved:gc
delete-branch(gc plans are also bound to the VCS head and manifest
hash),new's reuse/reset (plus a re-run of the bookmark-holder guard and
a same-workspace check),restore's reset,promote's source,repair's
refork (the branch must still be missing). A lazy fork resets an existing
branch only onto the headnewreviewed (workspace.toml
pending_resets), reuses a branch already at the pin or one a scope
sibling writes through, and otherwise refuses.gcplans are bound to the
digest of all visible history (a commit made in another workspace can
reference a pin the plan would release) and check every branch head before
the first action, so a stale plan does nothing at all;promote --rev
verifies a pin source still names the reviewed state; a forknewplanned
as fresh is re-checked for absence at apply, andnewrefuses outright
when the backend cannot list branches (unknown is not absent). commitcompensates as a unit. A failure after the pins -- manifest
write, listing, VCS commit -- releases only the pins this commit created
(never a reused one), restores the manifests and listings it wrote, and
journals the attempt as failed and rolled back. If the VCS commit landed
before the adapter raised, nothing is rolled back: history names the pins,
so the operation completes as a commit that succeeded and the trailing
error is surfaced (failed_after_commit).restoreandpromoteare closed over the branch scope. Restoring one
of several keys that write through one branch is refused (the message names
the siblings to include); with all named, the branch is reset once and the
siblingsshareit.promotefast-forwards a shared branch once instead of
once per key.- The writer lock also covers
add,remove,set,import,abandon,
forget-workspace, andupgrade;undojournals before it acts (a refused
undo is recorded as failed). Taking the lock re-readsworkspace.tomland
the manifests, so a long-livedReponever writes the state it loaded at
construction over what another process wrote since;snapshotwrites its
cache under the lock. undocompletes atomically: the undone mark rides in the done record (one
append), and a handler refusal that touched nothing ends the entry as a
failed attempt rather than leaving it started.promote KEY...refuses a subset that leaves unnamed siblings whose base
branch the write would move, whatever the source (a working ref, or a pin
by--rev), asrestoredoes.promotefast-forwards and merges from the state the plan reviewed, not
the source ref's current head: what lands is what was shown, whatever the
timing.ObjectBackend.mergetakesstr | Pin | Statelikepromote
(git, lakeFS, Dolt, and memory resolve a state to its commit). The inline
convenience methods (commit,new,promote,restore,gc,import)
plan and apply under one checkout lock, andnew/promote/restore/
importre-verify at apply (commitdoes not need to: a pin names the
captured state). The CLI's immediatetether commitgoes through
Repo.committoo; only--dry-run/--planbuild a separate plan.gcplans take the history digest before walking history, so a commit
landing during the walk stales the plan instead of slipping between the
references and the digest.restorechecks every head before the first reset and writes the
workspace after each one (with the siblings sharing the branch), so a kill
between two resets leaves each reset branch described as such.apply_committurns a planned key that is no longer registered into
StalePlanError(rolling back the pins it made before reaching it) rather
than aKeyError.promote --rev's scope closure and base-head check work from the kind and
locator the plan captured, so an object removed from the working tree since
the revision still refuses an unnamed sibling and a base that moved.snapshotandpullrun whole under the checkout lock: which refs to read
is decided from the workspace as it is on disk, not from the one a
long-livedRepoloaded.newwritesworkspace.toml-- bookmark set, every fork pending with the
reset it agreed to -- right after moving the VCS and before the first store
write, so a process killed in the fork fan-out leaves exactly a lazynew;
a fork that fails in the fan-out stays pending instead of being forgotten.gc: a branch that moved after the preflight (a race, not a stale plan)
is kept and reported while the rest of the plan finishes; a--force-prune
plan that could not read a head refuses at apply if the head reads now.- Pre-v4 manifests read from history resolve relative local paths against
the dataset root, the rule the migration applies to the working tree. promote's guarantee is stated as it is: a bookmark is planned whole or
not at all; once applying, each system's fast-forward stands on its own.set --pin recordon a pinned object takes effect at the next commit (the
pin is dropped;gcreleases the tag once no commit names it);--pin nativecreates one again. Before, the "unchanged" shortcut kept the pin.file:--file versionedmakes only a single remote object Addressable;
a recorded object state without a version id (unversioned bucket) is
refused by...
tether 0.1.0a8
Dataset namespaces: pins are tether.. and working branches tether.ws.....; gc never touches another dataset sharing a store. Operation log (.tether/ops.jsonl, per workspace) with tether ops, tether undo [ID | --to ID], and tether repair. tether upgrade brings a1..a7 datasets forward (renames refs in every store and rewrites history to match; fails closed). New entrypoints where the VCS half and the store half must agree: abandon (drops commits + the gc plan they free), restore KEY --from REV (per-object re-fork), forget-workspace. new --discard is required to reset a branch with uncommitted writes; a branch already at the pin is reused (no more Neon sibling per commit). status and ops flag dataset commits removed behind tether's back. Fixes: git pins fail when the remote push fails; Neon role out of identity; Neon API errors are BackendErrors; undeletable branches are planned as kept. BREAKING: [tether] version = 2 -- run tether upgrade on existing datasets.
tether 0.1.0a7
Fixes from the second review. States separate content from address (VOLATILE_KEYS / content_state): Iceberg metadata rewrites and Neon checkpoints no longer show as drift or mint duplicate pins; git dirty applies only to the checked-out ref. fork() onto an existing name resets it on every backend, Neon included (restore by head, sibling branch once pins hang off it). Stale detection is per object from recorded base states. Working-ref names carry a key digest and pin ids are 16 hex; earlier alphas' pins and branches are not recognised -- re-commit and new. apply_new verifies before moving the working copy and records successful forks before raising; git new keeps commits on a tether/ branch instead of a detached HEAD; gc --prune-workspaces discovers live jj workspaces and git worktrees. import locator changes drop the old working branch. export/publish/import and lakeFS/Dolt/DuckLake are labelled experimental.
tether 0.1.0a6
Lazy forking: tether new decides working branches and the first writable open creates them (--eager / [new] fork = "eager" for the old behaviour; --pin record objects still fork during new). tether promote: fast-forward each system's base branch to the fork, native three-way merge where the system has one (git, lakeFS, Dolt), refuse with a recipe otherwise; fork points recorded in workspace.toml (export schema_version 2). See CHANGELOG.md.