Skip to content
zfs backup with remote capabilities and mbuffer integration.
Branch: master
Clone or download
christogriffith and oetiker zetup: Don't expand variables in printf format
Expanding the timestamp option inside the printf format string results
in printf warnings because the % characters get interpreted as format
specifiers.

Fixes: #409
Latest commit defcf6d Mar 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bash_completion.d started basic bash completion (#242) Mar 17, 2017
bin zetup: Don't expand variables in printf format Mar 12, 2019
conftools release 0.19.0 Jun 19, 2018
doc release 0.19.0 Jun 19, 2018
init changes for macOS Startup (#358) May 15, 2018
lib Fixes #401 "Code review - destroySnapshots - check that $dataSet is a… Feb 2, 2019
man release 0.19.0 Jun 19, 2018
packaging/checkinstall Added zfsonlinux support Feb 7, 2018
t changes for macOS Startup (#358) May 15, 2018
thirdparty release 0.19.0 Jun 19, 2018
.gitignore changes for macOS Startup (#358) May 15, 2018
.travis.yml test on newer perls Apr 13, 2018
AUTHORS - znapzend now checks if pre and post snapshot commands can be execut… Jul 24, 2014
CHANGES Updated CHANGES Sep 24, 2018
COPYRIGHT release 0.19.0 Jun 19, 2018
LICENSE boom ... znapzend is real. thank you dominik! May 30, 2014
Makefile.am update list of files included in dist Jun 27, 2018
Makefile.in Makefile.in : regenerated updated EXTRA_DIST list Sep 24, 2018
PERL_MODULES Mojo::Log::is_level needs Mojolicious 6.47 Nov 4, 2018
README.md Added copr repository to README.md Nov 19, 2018
VERSION release 0.19.0 Jun 19, 2018
aclocal.m4 release 0.19.0 Jun 19, 2018
bootstrap.sh prepare for 0.13.0 Sep 18, 2014
configure Further fix for service integrations (fix evaluated vars other than P… Sep 24, 2018
configure.ac

README.md

ZnapZend 0.19.0

Build Status Coverage Status Gitter

ZnapZend is a ZFS centric backup tool to create snapshots and send them to backup locations. It relies on the ZFS tools snapshot, send and receive to do its work. It has the built-in ability to manage both local snapshots as well as remote copies by thinning them out as time progresses.

The ZnapZend configuration is stored as properties in the ZFS filesystem itself.

Note that while recursive configurations are well supported to set up backup and retention policies for a whole dataset subtree under the dataset to which you have applied explicit configuration, at this time pruning of such trees ("I want every dataset under var except var/tmp") is not supported. You probably do not want to enable ZnapZend against the root datasets of your pools due to that, but would have to be more fine-grained in your setup. This is consistent with (and due to) usage of recursive ZFS snapshots, where the command is targeted at one dataset and impacts it and all its children, allowing to get a consistent point-in-time set of snapshots across multiple datasets.

Complilation Inztructionz

If your distribution does not provide a packaged version of znapzend, or if you want to get a custom-made copy of znapzend, you will need a compiler and stuff to build some of the prerequisite perl modules into binary libraries for the target OS and architecture. For run-time you will need just perl.

On RedHat you get the necessaries with:

yum install perl-core

On Ubuntu / Debian with:

apt-get install perl unzip

On Solaris you may need the C compiler from Solaris Studio and gnu-make since the installed perl version is probably very old.

On OmniOS/SmartOS you will need perl and gnu-make.

On macOS, if you have not already installed the Xcode command line tools, you can get them from the command line (Terminal app) with:

xcode-select --install (or just install the full Xcode app from the Apple app store).

With that in place you can now utter:

wget https://github.com/oetiker/znapzend/releases/download/v0.19.0/znapzend-0.19.0.tar.gz
tar zxvf znapzend-0.19.0.tar.gz
cd znapzend-0.19.0
./configure --prefix=/opt/znapzend-0.19.0

If configure finds anything noteworthy, it will tell you about it. If any perl modules are found to be missing, they get installed locally into the znapzend installation. Your perl installation will not get modified!

make
make install

Optionally (but recommended) put symbolic links to the installed binaries in the system PATH.

for x in /opt/znapzend-0.19.0/bin/*; do ln -s $x /usr/local/bin; done

Packages

Debian control files, guide on using them and experimental debian packages can be found at https://github.com/Gregy/znapzend-debian

An RPM spec file can be found at https://github.com/asciiphil/znapzend-spec

For Fedora specifically there's also a copr repository by oranenj (sources at https://github.com/oranenj/copr-repos/tree/master/znapzend):

dnf copr enable oranenj/znapzend 
dnf install znapzend

Configuration

Use the znapzendzetup program to define your backup settings. They will be stored directly in dataset properties, and will cover both local snapshot schedule and any number of destinations to send snapshots to (as well as potentially different retention policies on those destinations). You can enable recursive configuration, so the settings would apply to all datasets under the one you configured explicitly.

Example:

znapzendzetup create --recursive\
   --pre-snap-command="/bin/sh /usr/local/bin/lock_flush_db.sh" \
   --post-snap-command="/bin/sh /usr/local/bin/unlock_db.sh" \
   SRC '7d=>1h,30d=>4h,90d=>1d' tank/home \
   DST:a '7d=>1h,30d=>4h,90d=>1d,1y=>1w,10y=>1month' root@bserv:backup/home

See the znapzendzetup manual for the full description of the configuration options.

For remote backup, znapzend uses ssh. Make sure to configure password-free login (authorized keys) for ssh to the backup target host with an account sufficiently privileged to manage its ZFS datasets under a chosen destination root.

For local or remote backup, znapzend can use mbuffer to level out the bursty nature of ZFS send and ZFS receive features, so it is quite beneficial even for local backups into another pool (e.g. on removable media or a NAS volume). It is also configured among the options set by znapzendzetup per dataset. Note that in order to use larger (multi-gigabyte) buffers you should point your configuration to a 64-bit binary of the mbuffer program. Sizing the buffer is a practical art, depending on the size and amount of your datasets and the I/O speeds of the storage and networking involved. As a rule of thumb, let it absorb at least a minute of I/O, so while one side of the ZFS dialog is deeply thinking, another can do its work.

Running

The znapzend daemon is responsible for doing the actual backups.

To see if your configuration is any good, run znapzend in noaction mode first.

znapzend --noaction --debug

If you don't want to wait for the scheduler to actually schedule work, you can also force immediate action by calling

znapzend --noaction --debug --runonce=<src_dataset>

then when you are happy with what you got, start it in daemon mode.

znapzend --daemonize

Best practice is to integrate znapzend into your system startup sequence, but you can also run it by hand. See the init/README.md for some inspiration.

Running by an unprivileged user

In order to allow a non-privileged user to use it, the following permissions are required on the ZFS filesystems:

Sending end: destroy,hold,mount,send,snapshot,userprop Receiving end: create,mount,receive,userprop

Troubleshooting

By default a znapzend daemon would log its progress and any problems to local syslog as a daemon facility, so if the service misbehaves - that is the first place to look. Alternately, you can set up the service manifest to start the daemon with other logging configuration (e.g. to a file or to stderr) and perhaps with debug level enabled.

In case you tinkered with ZFS attributes that store ZnapZend retention policies, or potentially if you have a severe version mismatch of ZnapZend (e.g. update from a PoC or very old version), znapzendzetup list is quite useful to non-intrusively discover whatever your current version can consider to be discrepancies in your active configuration.

Statistics

If you want to know how much space your backups are using, try the znapzendztatz utility.

Support and Contributions

If you find a problem with znapzend, please open an Issue on GitHub.

If you'd like to get in touch, come to Gitter.

And if you have a contribution, please send a pull request.

Enjoy!

Dominik Hassler & Tobi Oetiker 2018-06-19

You can’t perform that action at this time.