rsync-prelude

Fix moved/renamed files before running rsync to avoid useless retransmissions.

Features

• it respects the rsync filters (uses rsync to get the file list of candidate files)
• it works with remote hosts (unlike rsync-sidekick)
• it's plain Python 3, which is available basically everywhere
• it can detect circular moves/renames (a -> b, b -> c, c -> a)
• it avoids computing hashsums if the filesize is unique in the set of files (then it cannot be a moved file)

Install

• manually: download and run rsync-prelude
• using pip: python -m pip install git+https://github.com/gbabin/rsync-prelude.git

Prerequisites

• Python 3.7 or later

Usage

rsync-prelude [-h] [-v] [-p [WIDTH]] [-q] [-s] [-e RSH] [-f FILTER_ARGS] [--hash-tool HASH_TOOL]
              [--mv-cmd MV_CMD] [--cp-cmd CP_CMD]
              SRC DEST

positional arguments:
  SRC                   source directory
  DEST                  destination directory

options:
  -h, --help            show this help message and exit
  -v, --verbose         print internal actions and messages
  -p [WIDTH], --progress [WIDTH]
                        show current command (default width: 80)
  -q, --quiet           do not print actions taken
  -s, --script          just output script executing detected moves ('dry run')
  -e RSH, --rsh RSH     rsync remote shell command to use (default: ssh)
  -f FILTER_ARGS, --filter-args FILTER_ARGS
                        rsync args affecting file list
  --hash-tool HASH_TOOL
                        command to hash files. First word of the output must be the hash. It must
                        be available on both source and target host(s) (default: sha256sum)
  --mv-cmd MV_CMD       command used to move files (default: mv -n)
  --cp-cmd CP_CMD       command used to copy files (default: cp -n)

Example

function updatebackup() {
  LOCAL_DATA_DIR=~/myfiles
  RSH_CMD="ssh -p $BACKUP_HOST"
  RSYNC_FILT_ARGS="-aup -FF"
  RSYNC_ARGS="--info=progress2 --delete"
  rsync-prelude -v -e "$RSH_CMD" -f "$RSYNC_FILT_ARGS" $LOCAL_DATA_DIR $REMOTE_DATA_DIR
  rsync -v -e "$RSH_CMD" $RSYNC_ARGS $RSYNC_FILT_ARGS  $LOCAL_DATA_DIR $REMOTE_DATA_DIR
}

License

Distributed under the MIT License, see LICENSE.md.

Acknowledgments

Derived from rsync-prepare.