Rsync By Config
I'm unable to remember the
rsync options I usually need for transferring files to a remote location. Also, I'm too lazy to memorize the remote path. And of course I have a lot of different remotes with different directories. So I made a wrapper around
rsync which reads from a config file in the current directory.
Rsync By Config was previously called Simpler Rsync.
You can install Rsync By Config either with
pip install or manually. Using
pip is the recommended way (but it is not really tested beyond my OS X setups).
This all-important one-liner should take care of pretty much everything:
pip install https://github.com/AndiH/rsync-by-config/archive/master.zip
You should have a command
rbc now available in your shell. In the process, all dependencies should be installed as well.
Test it with
More manually but having the same effect, you could clone into some temporary directory and call
pip into that directory:
cd $TMPDIR git clone https://github.com/AndiH/rsync-by-config cd rsync-by-config pip install . rbc --help
cd ~/bin/ git clone https://github.com/AndiH/rsync-by-config ln -s rsync-by-config/rbc.py rbc.py --help
You should insert
~/bin/ into your
$PATH, e.g. via
export PATH=$HOME/bin/:$PATH so you can use
rbc.py globally from any folder. If
rbc.py does not run, make sure to have installed all dependencies. Note: For the time being, a message about this being not the ideal way to invoke rbc is printed. This will go away at some point.
rbc reads from a configuration file in the current directory, i.e.
.sync.toml. Depending on the parameters specified there, the content of a directory (e.g. the current one) is copied to a target location using
The Python script has a few command line options, all documented via
ENTRY: The name of the entry in the config file to be used for synchronization. If not specified, the default entry is taken. If there's no default, the first entry is taken. Maybe, at least, since the read-in of the config file is not strongly controlled… New in 3.0: Synchronize with multiple entries at the same time by specifying
entry,anotherentry(comma-separated, no space)!
--monitor: Run Rsync By Config in monitoring (or deamon) mode. This will monitor the source folder for changes and issue a synchronization if one occurs. The package Watchdog is used for this. (A more manual alternative to Watchdog is the command line utility
fswatch, which can invoke arbitrary programs when a folder is changed.)
--confg_file=somefile: Specify a different config file. The default is
--dryrun, preventing all actual copies. Good for testing.
--verbose: Output every step and test of the script.
--listhosts: Lists the hosts specified in the config file. Exits afterwards. Adding
--verbosewill list all options of the hosts specified in the config file.
rsyncas an additional option. Can be invoked multiple times. Shorthand:
rsyncoption is given in addition to the basic, default options hardcoded into the script file and in addition to the options defined in the config file.
To specify values for the options globally, environment values can be set. This is handy if you don't like my choice of calling the default config file
.sync.toml (see below) and want to change it. The command line options (at least the last three from the list above) are accessible as environment variables with a
RBC_ prefix, thanks to Click. So, for example, to rename the default config file, do:
export RBC_CONFIG_FILE=.my.toml rbc somehost
The config file used for Rsync By Config is written in TOML. One entry in the config file is for one distinct parameter configuration, i.e. usually for one remote location. The config file is structured as follows.
[firsthost] hostname = "first" target_folder = "/something/" rsync_options = ["--delete", "-v"] [secondhost] hostname = "127.0.0.1" target_folder = "/something/" default = true
The available keys are:
hostname: A hostname to be understood by
rsync. Hint: Use aliases in your
target_folder: The target directory to be syncing to (remote or local).
rsync_options: An array of strings of
rsyncoptions. They are used in addition to the default, basic options hardcoded into the Python program and the options supplied by the command line call.
default: A boolean (either
falseor not given) whether or not the current entry is the default. You yourself are responsible for preventing multiple defaults.
source_folder: Usually, Rsync By Config is expected to work from the current directory of invocation. Setting this value changes this behavior explicitly. Useful in combination with
gather, see next section.
local_folder are deprecated and will be removed soon™! There's a handy little command in the warning to convert config files.
Additionally to the parameters of an entry, global parameters true for all entries in the config file can be specified. The parameters need to be specified before the first entry occurs. Currently, only
rsync_options is supported. Example:
rsync_options = ['-v'] [firsthost] hostname = "first" # etc
Inverse Transfers (Gathering)
Rsync By Config also supports transfers from a remote host to the local machine (a gathering operation). An entry in the config file as an example looks like this:
[inversehost] hostname = "second" source_folder = "/some/target/" target_folder = "/some/source/" gather = true rsync_options = ["--update"]
gather option is the important one. It switches the order of source and destination in the underlying
Note: Due to the implementation, gather only switches the order of source and destination in the very last step. A result:
source_folder becomes the target of the
rsync transfer, and
target_folder the actual source. If this bothers you, let me know.
For copying on the same machine, Rsync By Config supports host-less operations between a source directory and a target directory. This mode is enabled if the
hostname key of the respective entry in the config file is missing. In this case, the script also checks for
target_folder to be a valid directory on the current machine. Example config file:
[localstuff] target_folder = "/dev/null/"
Some Python packages are required for Rsync By Config. All can be installed with
pip install sh toml click watchdog
While non-essential for its core task, Watchdog is needed for the monitoring capabilities. The dependency is optional, though.
When choosing the recommended way of installation via
pip all dependencies are installed in the process.