Skip to content

Releases: heyvaldemar/external-disk-backup

v1.1.1

Choose a tag to compare

@heyvaldemar heyvaldemar released this 22 Sep 14:12

Fixed

  • --verify honours BACKUP_RSYNC_EXTRA, so an exclusion stops reporting itself. The copy skipped what the setting told it to skip and the verification then called every one of those files missing from the backup. Excluding a media library's video while keeping its artwork, playlists and database — the case the setting exists for — produced a report of thousands of findings, all of them the setting working. Both halves now use the same list, and the suite holds them to it: the video stays out of the copy, and the verification does not mention it.

Upgrading

git pull. Nothing changes unless you set BACKUP_RSYNC_EXTRA; if you already had it set, --verify stops reporting the files you excluded.

Full history in CHANGELOG.md.

v1.1.0

Choose a tag to compare

@heyvaldemar heyvaldemar released this 14 Sep 17:03

This script has always written a last-ok stamp at the end of a successful run and never used it for anything.

It is the only thing that can tell a file the backup missed from a file that arrived after it, and without that distinction a comparison of the copy against the source is a list nobody can act on.

--verify walks the same BACKUP_SOURCES the copy walks — never a list of its own, because a verification that covers less than the backup answers a different question from the one being asked — asks rsync what differs, and judges each difference against the stamp:

what it sees what it means
older than the stamp, absent from the disk the backup missed it
older than the stamp, contents differ unexplained
newer than the stamp arrived since, counted in the totals and not named
no stamp at all nothing is a finding, and the run says why

Both of the errors in that table were made on the machine these rules come from. Treating every absence as explained produced a 221-line report that turned out to be a race between the 12:00 copy and the 13:00 check: every file in it was on the disk and identical. And with the stamp missing, every difference fell into "unexplained", so one lost state file would have reported a continuously-written log as disk corruption.

At the boundary it errs towards silence. The stamp has one-second resolution, and a file written in the second the backup finished could be either; counted as new, a genuinely missed file is excused for one run and caught by the next, when the cutoff has moved past it. Counted as missed, every such file is a false alarm, and an alarm that cries wolf is an alarm somebody mutes.

Seven scenarios cover each of the four verdicts, a configured source that never reached the disk at all, and the fact that a read-only pass does not move the last-run stamp a watcher reads to decide whether the backup is still running. Both boundary cases above were found by the suite rather than written into it.

external-disk-backup.sh --verify    # compares, copies nothing, moves no stamp

The default compare is size and mtime, the same question the copy itself asks. BACKUP_VERIFY_CHECKSUM=1 reads every byte on both sides instead — on the machine this comes from that is 1.3 TB and four hours, and a verification nobody can afford to run is a verification that does not run.

Upgrading: replace the script. --check and --verify no longer write the last-run stamp, which is a fix: a pass that copies nothing did not run the backup.

v1.0.0

Choose a tag to compare

@heyvaldemar heyvaldemar released this 05 Sep 05:15

Added

  • A marker file that has to live on the removable disk. An unmounted mount
    point is an ordinary empty directory, and a backup written there fills the
    disk it was meant to protect while reporting success. The run refuses without
    the marker, and says how to create it.
  • Versioned trash instead of a bare --delete. Everything deleted or
    overwritten moves to _trash/<date>/ and stays for a retention period. That
    is the difference between a mirror and a backup.
  • A functional probe of rsync itself. macOS ships openrsync, which accepts
    --delete and --backup-dir and silently ignores them, so every run would
    report success and produce a copy with no versioning. The script proves the
    behaviour on two temporary files and refuses to continue if the flags are not
    honoured. Not a version string: what the binary actually does.
  • Two stamps and a report that survives failure. last-run is written
    before any verdict, last-ok only after a clean one, so a watcher can tell a
    backup that keeps failing from one that stopped being scheduled. The report is
    always left on disk, because debugging a backup by running it again is a bad
    habit.
  • Parent directories created before the copy starts. rsync creates only the
    last component of a destination path and skips the source entirely otherwise,
    with one line of explanation — which is how eight volumes were quietly not
    copied on a first run.
  • Twelve end-to-end scenarios, most of them refusals, including one run
    against a deliberately crippled rsync so the probe is known to fire.