v1.6.1 — a false corruption flag no longer hides (or endangers) a good backup

A field incident on 2026-07-19 turned out to be three defects stacked on top of each other. A full backup was flagged with one corrupt frame affecting one file. Re-checked afterwards, all four shards verify perfectly clean — 168/168 frames on both the recorded SHA-256 and a full zstd decode. The flag was a false positive from a transient read, and everything downstream of it went wrong.

1. A corrupt full stopped being a cycle — and could be deleted as collateral

cycles() opened a new cycle only for a full set that is_complete, which forbids any corrupt frame. So the flagged full was reparented onto the previous cycle as though it were an incremental:

• it did not appear in the cycle list at all;
• the previous cycle still reported "complete" while concealing it;
• every incremental taken afterwards was shown under a full it was not computed against — the restore chain was misrepresented;
• and because Cycle.archives includes incr_sets, pruning the previous cycle would have deleted the later full with it. With max_cycles and four complete cycles present, that was one backup away from happening.

Grouping now keys on a new ShardSet.is_restorable — "every shard was written and can be read" — rather than is_complete — "...and nothing was found corrupt." The cycle it opens still reports is_complete == False, so retention keeps protecting it and it is never treated as a clean base. failed/empty/in-progress fulls still do not open a cycle: those are untrustworthy, not merely damaged.

2. Verify condemned a frame on a single read

verify_frame_checksums() read each frame exactly once, immediately after writing ~22 GB across four concurrent shard threads, with the page cache dropped so the read crosses the network. One transient bad read became a permanent corrupt record.

A suspect frame is now re-read — evicting just that range first — and classified:

Result	Verdict
A re-read matches the digest	read-recovered — stored bytes are fine, nothing condemned
Every read mismatches, identical bytes	Genuinely corrupt
Every read mismatches, different bytes	unstable-read — the read path is unreliable, so what is stored is unproven

Both unconfirmed classes are deliberately biased away from "corrupt" and reported loudly instead: a false flag demotes a healthy full out of its own cycle and does not self-clear, which is worse than an anomaly warning.

⚠️ verify_frame_checksums() now returns a 4-tuple (algo, count, bad, transient). Pass retries=0 for the old single-read behaviour.

3. A false positive could never be cleared

action_verify only printed — nothing ever wrote the manifest, so a bad flag was permanent short of hand-editing manifest.json, and while it was set the full stayed demoted.

--verify now reconciles: it clears a stale corrupt flag on a clean result, and marks a confirmed-corrupt archive that the manifest calls fine. It writes only on genuine disagreement, so a plain clean verify still touches nothing — and a system-class plan escalates only in that case, never for a no-op. The prior state is recorded in the entry's notes rather than silently overwritten.

Caveat: an entry marked corrupt loses whether it was ok or ok-with-warnings beforehand; it is restored as ok, with the prior state noted.

Verified end to end

On the real archives: all four shards OK, manifest reconciled, the cycle restored with its own incrementals, audit trail intact —

verify: manifest updated —
  2026-07-19_full.s3of4.pax.zst: corrupt -> ok (corrupt, 1 corrupt frame(s))

Also: the README install example had been telling people to install v1.4.3.

388 tests pass (13 new).

v1.6.0 — privileged restore to original locations, and honest verify state

Restore to original locations now escalates instead of silently failing

Restoring a system-class plan's files back to where they came from needs root, but that path had no pkexec support at all. The GUI's Original location… button pointed an unprivileged extract at /, so anything outside your own home failed with permission errors partway through — leaving a half-finished restore on disk. The workaround was to extract to $HOME and copy into place with sudo: fine for a sysadmin, not discoverable for the audience the GUI exists for.

• New privileged helper /usr/libexec/timetraveller-restore-system-files with a matching com.timetraveller.restore-system-files polkit action, same allow_active=auth_admin_keep posture as the other four.
• The helper's destination is hardcoded to / and is not a parameter. That is load-bearing: accepting a caller-supplied --into would turn it into a write-anywhere-as-root primitive, and any other destination is somewhere you could have written to yourself. It takes no archive path either, deriving the archive directory from the root-owned /etc/timetraveller/<plan>.yaml. Every member must be ./-rooted (which doubles as argv-injection defence), with no .. component, no NUL, no control characters, and capped count and length.
• Privileges are pre-flighted before any bytes are written, in both the GUI and the CLI. Routing is decided by the destination, not the plan class, so extracting a system-plan member into ~/Restored stays unprivileged and prompt-free.
• A root-owned destination that is not / is refused rather than escalated — since the helper forces /, routing e.g. --into /opt/foo through it would have scattered files to their original locations instead of the directory you asked for.
• Source mode (Restore from location…) gets no automatic escalation, because the helper takes its archive directory from the root-owned plan config and never from a caller-supplied path. Documented rather than silently absent.

Two ways the tool could report success it had not earned

The naive (no-sidecar) extract path. It discarded tar's stderr, ignored its exit status entirely (the check was literally pass), then tallied results by walking the destination directory — so a wholly failed extract into a non-empty directory returned a large bytes_written and read as a clean restore. It now honours tar's exit status, distinguishes a genuinely benign "Not found in archive" from a real failure, and counts only the members tar reports having written.

⚠️ Behaviour change: extract.extract_files can now raise extract.ExtractError where it previously returned stats describing a failure as a success.

verify-after-write. It returned a bare [] for four different outcomes — verified-clean, the check threw, there was no checksum-bearing sidecar to compare against, and verification was switched off — so corrupt_frames=0 was unfalsifiable and a backup nobody verified rendered identically to a verified one. The only distinguishing signal was a stderr warning that never reached the log file, and cron entries don't redirect stdout either, so on a scheduled run it left no trace anywhere.

• New ArchiveEntry.verify_state: verified / unverified / error / disabled. Additive and optional, so old manifests load unchanged.
• The Archives list shows "ok (unverified)" in muted grey with a tooltip explaining which case applied. Archives written before v1.6.0 carry no verify state and are left unmarked — greying out all history would be noise, and the distinction self-heals from the next backup onward.
• It deliberately does not feed status or is_complete. An unverified shard is a gap in knowledge, not damage; treating it as corruption would mark healthy cycles incomplete and change what retention prunes.

Logs stop disappearing

/var/log/timetraveller is now owned by the package, so dpkg creates it on install and recreates it on upgrade. It previously existed only because the worker mkdir'd it at runtime, which meant an OS reinstall took the backup logs with it. Verify and corruption diagnostics are now written to the shard's log file rather than only to streams nothing captures.

367 tests pass (42 new).

v1.5.4 — pkexec dependency fix (important for system backups)

This release fixes a packaging bug that can leave all privileged operations non-functional on a clean install — system backups, archive deletion, maintenance, crontab install, and system config writes.

The bug

debian/control declared polkitd | policykit-1. Since Debian 12 / Ubuntu 23.04 the old policykit-1 package was split, and polkitd ships only the daemon — /usr/bin/pkexec is now its own binary package.

A minimal install could therefore satisfy every declared dependency and still have no pkexec, breaking every privileged helper path. Existing installs often had pkexec pulled in by a desktop environment and never noticed.

Fixed: Depends now includes pkexec | policykit-1 — the alternation also covers older releases where policykit-1 still shipped the binary.

Also added to Recommends: a graphical polkit authentication agent. This is Recommends rather than Depends because GNOME's agent lives inside gnome-shell and Provides: nothing, so no alternation can express "this desktop already has one". pkexec falls back to its own textual agent for CLI use regardless.

Archive provenance fix

timetraveller/__init__.py still said 1.5.0 while the changelog and pyproject.toml said 1.5.3. This was not cosmetic: worker.py stamps archives with created_by=f"timetraveller {__version__}", so every archive written by v1.5.1 through v1.5.3 falsely claims it was created by 1.5.0.

Existing archives keep the wrong stamp; archives written from v1.5.4 onward are correct. The version lives in three places, not the two the release process had assumed.

If escalation still fails after upgrading

If you administer the machine remotely (RDP/SSH), polkit may deny outright with no password prompt, which looks like escalation is missing rather than refused. The shipped policies set allow_inactive=no, and remote logind sessions report Remote=yes with an empty Seat=. Diagnose without triggering a prompt:

pkcheck --action-id com.timetraveller.run-system-backup --process $$
# exit 1 = flat deny (inactive/remote session);  exit 2 = challenge, a prompt would appear

To allow it for admins, add /etc/polkit-1/rules.d/49-timetraveller-remote.rules:

polkit.addRule(function(action, subject) {
    if (action.id.indexOf("com.timetraveller.") !== 0) return polkit.Result.NOT_HANDLED;
    if (!subject.isInGroup("sudo")) return polkit.Result.NOT_HANDLED;
    return polkit.Result.AUTH_ADMIN_KEEP;
});

This still requires a password — it only lifts the remote-session restriction.

Install

sudo apt install ./timetraveller_1.5.4_all.deb

v1.5.3 — Help & README: search + restore-from-anywhere

Documentation release. The cross-archive search panel and Restore from location… (restore-from-anywhere) had shipped without any in-app Help or README coverage — both are now documented. The help text is compiled into the GUI, so this ships as a release rather than a docs-only push.

• How to search across archives — the 🔍 Search files… panel: filename vs. full-path matching, per-path version lists, jump-to-tree, and extracting the selected copy straight from the results. Extract acts on whichever pane is in view (the v1.5.1/1.5.2 behaviour).
• Restoring from a drive with no plan configured — browse, search, and extract from a USB drive or NAS share via Restore from location…, with no local config.
• README status line corrected (1.4.3 → 1.5.3) and two feature highlights added.

No code or behaviour changes from v1.5.2.

Install: sudo dpkg -i timetraveller_1.5.3_all.deb

v1.5.2 — Archives browser/search cleanup

Cleanup pass on the Archives browser/search. Three fixes:

• Source-mode search/browse divergence. In restore-from-anywhere (source) mode the cross-archive search read the local sidecar mirror while the browse tree read the browsed directory, so their results could disagree. Search now reads the same sidecar source the browse tree uses per mode, and stats the sidecar file rather than trusting the manifest's (possibly stale) has_sidecar flag.
• Extract button matches the visible selection. The bottom "Extract selected…" button acts on the browse tree, but stayed visible while the search page (which has its own extract button) was shown. It's now hidden while searching, so the only exposed extract always matches what's on screen.
• Blank file tree after search. Opening an archive and switching to search before its sidecar finished loading set the tree model on a hidden stacked-widget page, leaving the rows un-laid-out — so the browse pane came back blank and wouldn't repopulate. The view is now relaid out when the browse page becomes visible again, and blanking the tree clears the current-set guard so a reselect always reloads.

Install: sudo dpkg -i timetraveller_1.5.2_all.deb

v1.5.1 — extract selected files from the Archives search results

GUI fix: the Archives search panel now has its own 'Extract selected…' button. Previously the bottom Extract button always acted on the browse file tree, so selecting a file in search and extracting could silently pull the wrong file (e.g. asking for ./etc/hosts and getting ./etc/brltty). Selecting a version row extracts that exact copy; a path row extracts its newest version.

TimeTraveller v1.5.0

Post-restore hardening — integrity, recovery, and restore-from-anywhere. Motivated by a real bootloader recovery and forensics showing 2 isolated bit-flipped frames in 17,732 on a home full (a non-ECC single-event-upset signature).

Integrity: verify-after-write (D1)

With verify_after_write (default on), each shard re-reads itself from the backup store immediately after writing — cache-dropped, so it checks what actually landed, not the client buffer — inside its own writer thread so the read overlaps sibling-shard writes (near-free in wall-clock). A frame that fails its checksum marks the shard corrupt, records the affected files, keeps the archive browsable, skips retention (preserving the redundancy heal needs), and exits with a --heal hint. Catches post-write corruption (e.g. non-ECC bit flips) the same night instead of years later at restore.

Recovery: heal-from-redundancy (D2)

--heal <backup> finds files damaged by frame corruption and restores a clean copy of each from another cycle that still holds one; unrecoverable files are reported. The GUI gains a corrupt badge + Recover damaged files (heal)… action.

Portable restore-from-anywhere

• A small timetraveller.restore.json descriptor (plan identity + original source roots) travels beside each manifest, backfilled on --refresh-from-mount.
• A self-contained timetraveller-restore.sh recovers on a machine with no TimeTraveller installed (needs only bash, python3, tar, zstd).
• New Restore from location… in the GUI: browse to a backup on a USB drive, external disk, or network share and restore directly, with no matching plan config. Extract to the original location (behind an overwrite warning) or a staging directory.

Install / upgrade

sudo apt install ./timetraveller_1.5.0_all.deb

v1.4.4 — browse GUI-run system backups + manual refresh-from-mount

A small but important follow-up to v1.4.3, from real-world testing.

🐛 Fix — browsing a GUI-run system backup

After a system/homes backup (or delete/reindex/recover) run from the GUI, the Archives tab would list the backup but couldn't browse it ("no sidecar in local mirror"). Cause: those operations run as root via pkexec, and the per-shard sidecar files landed in root's local mirror, not yours — only the manifest was synced back. Now the sidecars are synced too, so a GUI-run system backup is immediately browsable.

✨ New — manual "Refresh from mount" escape hatch

A new Archives right-click → "Refresh from mount (rebuild local mirror)" rebuilds your local manifest + sidecar mirror from the backup mount on demand — for any time the local view drifts from what's on disk. It's reachable even when the list is empty (right-click empty space), is read-only on the mount, and never needs root. (Equivalent to --list-archives --refresh-from-mount on the CLI.)

📦 Install

sudo apt install ./timetraveller_1.4.4_all.deb

290 automated tests pass. Recommended for anyone on v1.4.x who backs up system-level plans.

v1.4.3 — Archive integrity + system-plan GUI fixes

A stability + integrity release. Adds verifiable archive integrity and closes the last gaps in managing system-level backups from the GUI.

✨ Archive integrity (new)

• Per-frame SHA-256 checksums, computed inline as each archive is written — effectively free (no extra read pass; the streaming single-read write path is preserved). Recorded in the .frames.json sidecar (schema v2).
• --verify reworked into a fast, decompress-free integrity check: it re-reads each frame and compares its SHA-256, naming the exact corrupt frames. Shard-group aware; handles quarantined .failed archives; older archives fall back to a full decompress.
• zstd's own per-frame content checksum is now enabled too, so every restore / browse / recover self-verifies automatically.
• Motivation: catching real write-path corruption (8 corrupt frames in a 214 GB shard) that exit-code classification alone could not see. See docs/design/archive-integrity.md (including the non-ECC-RAM limitation).

🛠 Fixes

• System-class plans (system / homes) now work fully from the GUI as a normal user. Delete, Run full/incr now, reindex, and recover-failed all escalate to root via pkexec (5 audited helpers + polkit policies, each validating a plan allowlist and tightly-scoped args) instead of failing with permission errors on root-owned archives.
• No more false failed backups from benign file-race warnings — e.g. a browser rewriting its own cache mid-backup (a file changed as we read it co-occurring with a vanished file no longer trips the whole run).
• apt remove cleans up after itself — TimeTraveller's managed cron blocks are stripped from root's crontab and any user crontab on uninstall.

📦 Install

sudo apt install ./timetraveller_1.4.3_all.deb

Per-version detail (v1.4.0 → v1.4.3) is in debian/changelog. 288 automated tests pass.

⚠️ Testing release — please report issues. Backups written by this version are forward-compatible; the new checksum sidecar is additive.

TimeTraveller v1.3.0

First public test build since v1.1.1 — bundles three releases (v1.1.2, v1.2.0, v1.3.0). Thanks for helping test!

What's new since v1.1.1

Archive management (v1.2.0)

• Delete from the GUI. Right-click a cycle or a backup in the Archives tab → Delete cycle… / Delete backup….
• Type-to-confirm safety gate. The dialog shows the blast radius (shard files, bytes freed, dependent incrementals) and you must type the target's identifier; the Delete button stays disabled until it matches. The newest complete backup and incremental-bearing fulls are flagged.
• Reindex / Recover are now also in that right-click menu (previously easy-to-miss buttons).
• New CLI: --delete-cycle CYCLE_ID, --delete-set GROUP_ID (refuse the newest complete backup / dependent fulls unless --force).

Self-describing archives & export (v1.3.0)

• Each shard now writes a <archive>.meta.json sidecar (no extra read of the archive — the single-read write path is preserved).
• Manifest rebuild: --list-archives --refresh-from-mount reconstructs a lost on-mount manifest from the .meta.json sidecars.
• Export bundle: right-click → Export cycle… / Export backup… (or --export-cycle / --export-set --into DIR) copies a whole logical backup + all sidecars + a manifest slice, group-atomically (won't write a partial set).

Sharding control (v1.1.2)

• The plan editor now has a Parallelism control to set write-stream count (Auto / explicit).

Install

```
sudo apt install ./timetraveller_1.3.0_all.deb
```
(Architecture: all, pure-Python; needs python3 + PyQt6 + zstd.)

What I'd love feedback on

• The delete flow — is the type-to-confirm clear and not annoying?
• Discoverability of the right-click menu (delete/export/reindex/recover).
• Export → copy to another machine/removable → does it browse/restore there?

⚠️ Testing caution

Delete is permanent. Please test destructive actions against scratch/throwaway backups first, not your only copy of anything important.