Skip to content

residue v0.0.7

Choose a tag to compare

@github-actions github-actions released this 07 Nov 22:35
· 88 commits to main since this release
v0.0.7
f73cb3b
  • Automatic cleanup in one command (#28): residue service install and residue watch take no ROOT. Without one they use the usual code folders in the home folder that exist: ~/src, ~/code, ~/Code, ~/dev, ~/Developer, ~/projects, ~/Projects, ~/repos, ~/git, ~/github, ~/GitHub, ~/work, ~/Work and ~/workspace, each once (on a case-insensitive filesystem ~/code and ~/Code are one folder), and print which. Never the home folder itself, which holds far more than code, and nothing under Documents, Desktop or Downloads, which macOS guards with a permission prompt a background service cannot answer. When none exists, both print one line that names the folders and how to pass a ROOT, and exit 2. A ROOT replaces the list. service install never prompts, so an installer can run it, and prints the folders and residue service uninstall to turn it off.
  • service install writes the resolved roots into the service, so it watches the same folders at every start, and marks them as the default ones. service status now shows what the service runs (runs:) and which folders it watches (watches:). Running service install again replaces the service: without a ROOT it keeps the ROOTs a user named for the service it replaces, dropping any that is no longer a folder, with a note, and a service on the default folders looks for them again, which picks up a folder created since. Options are not kept: each one not given goes back to its default, and the command says so when the service it replaced was a --dry-run. The Homebrew formula defines no service. If a Homebrew service for residue exists anyway, from a formula of the user's own, service install says so and exits 1, since only one watch runs per user, and service status points to brew services info residue.
  • Decision (#28), watch fetches. Until now residue never fetched, so watch saw a landed branch only after the user's own git fetch --prune, and an automatic cleanup could wait for one indefinitely. watch now runs git fetch --prune --no-tags for each repository about every 30 minutes (--fetch-every DURATION; each repository on its own clock, its interval varied by up to a tenth). It fetches only the remotes that a linked worktree's branch tracks, and nothing in a repository with no such branch, since no other ref decides anything watch does. It fetches in the main checkout with --no-write-fetch-head, --no-auto-maintenance and --no-recurse-submodules, so no file in a worktree's Git directory, which is the idle clock, changes. It never waits for a person: a new session with no terminal, GIT_TERMINAL_PROMPT=0, SSH_ASKPASS_REQUIRE=never, GCM_INTERACTIVE=never, and SSH BatchMode=yes unless the user set GIT_SSH_COMMAND, GIT_SSH or core.sshCommand, which is used as it is; and a five-minute deadline, after which the fetch and every program it started (SSH) are killed. A fetch only refreshes the remote-tracking refs that the scan and the removal read: it is never itself a reason to remove, every check still decides, and a failed fetch logs one line (once, until it changes or works again) and leaves that repository's refs as they were. A branch the fetch finds deleted is logged before any removal it leads to. --no-fetch turns it off; --dry-run still fetches. scan, explain and remove never fetch, so the removal child's three upstream reads stay local. The AGENTS.md invariants record this.
  • Upgrading: a service installed by 0.0.6 or earlier carries neither --fetch-every nor --no-fetch, so after the upgrade it fetches every 30 minutes. To keep the old behavior, install it again with --no-fetch and its ROOTs.
  • residue scan at a terminal ends with one line when automatic cleanup is off, that is when no residue service or Homebrew service is installed and no watch of this user is running: Automatic cleanup is off. Turn it on: residue service install, with the scanned PATHs added when the default folders do not hold every repository it listed. Only on macOS and Linux, only when it lists linked worktrees, never in a pipe or with --json; the JSON schema is unchanged.
  • residue scan --json (#45): each repository's free_space has device, the filesystem's device number (st_dev) as a decimal string, so a reader can show one free-space figure per disk, as watch decides per disk. A string, since a device number can exceed 2^53, past what a JavaScript number holds exactly. It groups repositories within one document only: it changes across reboots and mounts, so never store it or compare it across runs. The schema stays residue.scan/v1.
  • A child process that overruns its deadline and leads its own process group, as the fetch does, is killed with its group.
  • Decision (#31), every deletion is restorable. Until now only remove --trash could be undone; a worktree that remove, --no-trash, watch or the trash purge deleted was gone, and its HEAD commit was safe only while a branch or reflog held it. Now each of them first writes a removal record, residue-removed/ENTRY/info in the repository's Git common directory (the path, branch, HEAD, upstream, why, by which command (by remove, by sweep, by watch; a purge names the run that purged), when, the size and the ignored entries the deletion took, plus copies of the worktree's HEAD reflog and config.worktree), and a ref, refs/residue/removed/ENTRY, at that HEAD, so a deleted branch, an expired reflog or git gc cannot lose the commit. A removal writes them after the move aside and before its check at the new path, and drops them when that check fails; the purge writes them after its re-check. Both write them before git worktree remove; if any of it cannot be written, it deletes nothing (could not write its removal record, so residue deleted nothing, result skipped, exit 3; a purge keeps the entry in the trash). info is written under another name, synced and renamed into place, the record's directories are synced, and the refs are written in one git update-ref --stdin transaction with core.fsync=reference and core.fsyncMethod=fsync (Git 2.36 adds reference to its platform default; macOS's default method only writes out), so a crash leaves all of it or none.
  • Decision (#31), commits only the worktree's own reflog held are kept. git worktree remove deletes the worktree's Git directory and its HEAD reflog with it. A commit made on a detached HEAD and then checked out away from, or the commit before an amend or a rebase, could have that reflog as its only holder, so git gc deleted it after the removal although every check passed. Now the record pins those commits: one ref each, refs/residue/reflog/ENTRY/N, at the fewest commits that reach all of them (git merge-base --independent over every commit the reflog names, as an old or a new value, that no ref outside refs/residue/ and no worktree's HEAD reaches). Another reflog naming such a commit does not count as a holder: a branch's reflog goes with git branch -D, and reflogs expire. A reftable repository (Git 2.45 and later) has no logs/HEAD file to read or copy; there residue asks Git for the commits that reflog names (git rev-list --walk-reflogs HEAD, each entry's new value) and pins those, and since restore cannot write that reflog back, the record stays until --keep-removed. The worktree's private refs (refs/worktree/, refs/bisect/, refs/rewritten/) and an operation in progress already keep it from removal, so the reflog is all its Git directory adds. Removal does not wait on this; it pins and goes on. The invariant, in AGENTS.md: while a record lives, nothing residue removes, restores or forgets makes such a commit unreachable.
  • What --keep-removed can lose: once a record is forgotten, a commit only its refs held is left to git gc. That is the commits of a branch deleted after the removal, for example after a squash merge, and of work only the worktree's reflog held. For those, --keep-removed (30 days) is the only grace period. The README says so. Decision, a ref and not a branch: a branch would show in git branch, could be checked out, pushed or deleted by the user's own tools, and its name could collide; refs/residue/ is residue's alone. git log --all and git push --mirror see it.
  • residue restore lists trash entries and removal records together, newest first, with their kind and why they went, and residue restore ENTRY recreates a removed worktree with git worktree add at its path, never --force: on its branch when the branch is at the recorded HEAD or ahead of it; as BRANCH-restored at the recorded HEAD when the branch was rewritten, which stays as it is; on the branch created again when it was deleted; detached when it was. A branch checked out in another worktree, or anything at the path, refuses; --to PATH restores somewhere else. The ignored files the deletion took, such as node_modules/, are listed as not kept. A removal that did not finish, whose worktree Git still lists under its hidden name, is listed as interrupted, and restore moves it back; when Git deleted that worktree's HEAD reflog or config.worktree, restore puts the saved ones back, and when Git can no longer use it, restore says to delete it and run git worktree prune, after which the same restore recreates it. A recreated worktree gets the saved HEAD reflog back, in front of its new one. Then restore drops the record and its refs, whether it recreated the worktree or moved it back, but only once the worktree's HEAD contains the recorded HEAD and its own HEAD reflog names every pinned commit; when it could not put back the saved config.worktree or reflog, or a branch moved while it ran, the record stays, and restore says where and why. An ENTRY path is looked up from the nearest folder above it that exists, and when that finds no entry, from the current directory; but never from the home folder or a folder above it, which would read every folder in it (on macOS, a permission prompt for Documents, Desktop and Downloads): restore asks to be run inside the repository instead.
  • Every remove run forgets the removal records older than --keep-removed DURATION (default 30d) in the repositories it covers, in the plan you confirm: the record and its refs, in one transaction, only while each still names the commit the record names; with any other ref under its name, or one that moved, the record and its refs stay. watch's child uses the default. A directory with no info and no ref, left by a run stopped while it wrote one (nothing was deleted then), is cleared once it is an hour old.
  • A removal record residue cannot read or forget never fails a removal: remove reports it on stderr (residue: warning: ...) and in a new JSON array, warnings, and the exit status says only whether the PATHs and the trash went. errors keeps its meaning: what residue could not read about the PATHs, the repositories and the trash. watch logs both. Otherwise one damaged record would have made every later remove in its repository exit 3, against "exit 3 means it is still there".
  • A deletion that fails after Git deleted part of the worktree keeps its record. git worktree remove deletes the files, then Git's directory for the worktree, entry by entry in the order the filesystem lists them, and stops at the first one it cannot delete; so it can fail with the worktree still listed and its files, its HEAD reflog, its config.worktree, its HEAD or its commondir gone. residue undid the record whenever the files were still there, so after a failure in Git's directory it moved the worktree back without the reflog and dropped the refs that kept the commits only that reflog held, and git gc deleted them. Now it undoes the record only while nothing of the worktree is gone: Git lists it and can still use it, its files and .git file are there, its HEAD contains the recorded HEAD, its logs/HEAD still begins with the saved copy and its config.worktree is the saved copy. Otherwise the worktree stays under its hidden name, never moved back, and the failure names the record (Git still lists it at PATH, but part of its HEAD reflog is gone; its removal record keeps its HEAD and reflog: residue restore RECORD). The purge does the same.
  • Entry names are unique across the trash and the removal records: both are NAME-STAMP, and a new one skips a name the other kind has. Restore's JSON reads worktree_config and head_reflog from the record as it was listed, not from the live files.
  • Changes from 0.0.6: an ENTRY that names more than one entry (the same entry name in two repositories, or a worktree name or branch that matches a trash entry and a removal record) is an error that lists them and asks for the entry's path or name; 0.0.6 took the first. remove --yes PATH now also forgets the removal records older than --keep-removed in that repository, as it already purged the old trash entries.
  • A worktree you restore is never removed again because its work landed, by watch or by remove --only-if-upstream-gone, and watch no longer fetches for its branch: it was wanted back after it landed. Disk pressure, remove PATH and a sweep still remove it. The mark is a file, residue-restored, in the worktree's Git directory; it counts as a change for --idle.
  • Every removal line ends with the command that brings it back: removed ... ; restore: residue restore RECORD from remove, and the same in the watch log, which also logs the records it forgot. scan counts each repository's records (removed: N restorable) and adds or: residue restore RECORD to an interrupted removal that has one; explain shows restored for a restored worktree.
  • JSON, additive, schemas stay at v1. Remove: results[].removal_record, purged[].removal_record, removal_records (reason, by, keep, keep_seconds), forgotten, and warnings, the removal records it could not read or forget (errors is unchanged). Restore: restore (kind, entry, path, branch, head, checkout, moved_from, new_branch, branch_tip, not_kept, notes), removed (each record with by, head_reflog, reflog_refs and incomplete among its fields), and kind on each trash entry. Scan: each repository's removed and summary.removal_records. Each worktree in scan, explain and remove: restored and restored_error. New options: --keep-removed, --to, and --removal-reason watch-landed|watch-low-disk, which watch passes to its child.
  • On illumos, purging one trash entry while others remain no longer reports a failure after the worktree was deleted, and restoring one no longer adds a note that its record could not be removed. illumos reports a directory that still has entries as EEXIST, not ENOTEMPTY, and residue now treats both as "not empty yet". Forgetting a removal record uses the same check.
  • Tests (#31): scripts/contract-check.sh gains 18 cases: the record and ref a removal writes (with by remove) and a restore from them, in JSON and text; a commit only the worktree's reflog held surviving the removal and git gc --prune=now, then named again by the restored worktree's reflog (fails on the first #31 build: no pin); the commit before an amend, which the branch's reflog also named, pinned and surviving git branch -D and git gc --prune=now, then named again by the restored worktree's reflog (fails on the second review build, 69c2d73: no pin); the lone commit again in a reftable repository, pinned with no logs/HEAD file and surviving git gc --prune=now, the record staying after the restore (skipped where Git cannot make a reftable repository); an unreadable record, a fresh and an hour-old entry with no info next to a removal that exits 0 with the warning in warnings and nothing in errors (fails on the first #31 build: exit 3; on 69c2d73: the warning in errors); every remove document has a warnings array; nothing deleted when residue-removed or refs/residue is a file; --keep-removed forgetting only the expired record; restore with the branch ahead, rewritten, deleted, checked out elsewhere and detached, and with the path occupied, then --to; and three new usage errors. scripts/recovery-check.sh gains a case where the run stops after writing its record and restore moves the worktree back, and the cases check how many removal records and refs each leaves (the stopped one, through that restore). It also gains two cases where git worktree remove fails after deleting the worktree's HEAD reflog, or its HEAD, from Git's directory: the worktree stays under its hidden name, the record and both refs stay, and the commit only the reflog held survives git gc --prune=now; restore then moves it back with every file, or, with the HEAD gone, says to delete it and run git worktree prune, after which restore recreates it (both fail on 69c2d73, which undid the record and moved the worktree back). .github/smoke.sh restores the worktree it removed. .github/watch-smoke.sh case 8: a watch removal's log line names the restore command, its record says by watch, the restore brings the worktree back, a fetching watch keeps it past --grace after its branch landed, and disk pressure still removes it. In-crate tests: the record's round trip (and a record without by still reads), escaping, entry names and records missing what restore needs.
  • Tests (#28): .github/watch-smoke.sh case 7, run in a scratch home folder: watch with no ROOT uses ~/src; with --no-fetch a worktree whose branch was deleted on the remote without a local fetch stays; watch's own fetch prunes the ref, writes no FETCH_HEAD, and the worktree is removed; a remote it cannot reach is logged and its refs and worktree stay; a home folder with none of the folders is the usage error. It runs on NetBSD, OpenBSD and DragonFly too, expecting no removal. Case 6 passes --no-fetch, since it sets refs by hand. .github/service-smoke.sh installs with a ROOT, again without one (the ROOT is kept), and after an uninstall without one (the default folders, marked as such in service status). In-crate tests: the ROOTs service status reads back from what to_args wrote, the recorded fetch setting, --no-fetch with --fetch-every refused, and every default folder named in residue watch --help.