Simple ZFS backup from one pool to another via sending snapshots, deleting old ones in time machine style. I'm using a Mac with TensCompliments ZFS implementation.
How it works
- the script creates a snapshot on the source pool every time it is called.
- then it figures out the last snapshot on the destination pool that matches to one on the source pool.
- it sends the snapshot from the source to the destination.
- removes old snapshots on the source - it keeps just n-snapshots.
- removes old snapshots on the destination - time machine fashion : 5min/last day, 1 hour last week, 1 day last 3 months, 1 week thereafter
It requires perl and the Time::Local and Date::Parse libraries. If you are on a Mac you can install them by using the command line:
$export PERL_MM_USE_DEFAULT=1 ; perl -MCPAN -e 'install Date::Parse' 'install Time::Local'
If you are on a different OS (like linux or bsd) everything should work. The checkbackupscript can't find out the last sleep and boot time then and will bug you about backups beeing too old when the machine has beeing powerd off for some time.
How to use
start the script from the command line with --sourcepool and --destinationpool options.
$ zfstimemachinebackup.perl --help [zfstimemachinebackup.perl] module options are : --configurationfilename (string) default: config.ini current: not used as Config:IniFiles module not present --createdestinationsnapshotifneeded (flag) default: 1 --createsnapshotonsource (flag) default: 0 --debug (number) default: 0 --destinationhost (string) default: --destinationpool (string) default: ocean/puddle --help (option) default: current: 1 --replicate (flag) default: 0 --snapshotstokeeponsource (number) default: 0 --sourcepool (string) default: puddle
Set --recursive=1 if you want to send the pools and all sub pools recursively.
Set --createsnapshotonsource if you want to create snapshots on the source Unset --createdestinationsnapshotifneeded=0 if you don't want the destinationpool to be created.
My current setup looks like this:
$ zfs list puddle 181Gi 19.3Gi 175Gi /Local ocean 708Gi 911Gi 371Ki /Volumes/ocean ocean/puddle 167Gi 911Gi 163Gi /Volumes/ocean/puddle
/Local is where my home directory lives. The script is called as follows
$ ./zfstimemachinebackup.perl --sourcepool=puddle --destinationpool=ocean/puddle --snapshotstokeeponsource=100 --createsnapshotonsource
So puddle is set as source, ocean/puddle will receive the snapshots from puddle and 100 snapshots are kept on puddle itself.
If you want to change the times when backups are removed on the destination you can change the following hash:
my %buckets = ( 1 *24*3600 => 5*60, # last day every 5 minutes 7 *24*3600 => 3600, # last 7 days, every hour 90 *24*3600 => 1 *24*3600, # last 90 days, every day ); my $buckettime = 7*24*3600; # keep weekly backups for beyond the time specified in %buckets.
For some reason the autoscrub feature of tens complement does not work for me, so I added a script you can use to my ZFS Timemachine script.
usage: ./autoscrub.perl --scrubinterval=14
will scrub your pools every 14 days. If you cancel a scrub that will be recognized but also it will be scrubed after the scrubinterval passed , in case you forgot that you canceled it.
You can start it for different pools as well.
I'm using it in a crontab entry: 1 * * * * cd ~jolly/Binaries/ZFSTimeMachine;./autoscrub.perl >/dev/null 2>&1
The checkbackup.perl script checks if your backupscript is working correctly. As I do sleep my machine it will check if the snapshots are beeing done within the last 2*snapshotinterval+snapshottime seconds since the last wake or reboot. Exit code is correct depending if the snapshot is there or not.
It has three options :
--pools which pool(s) to use comma separated list --snaphotinterval how often do you create snapshots --snapshotime how long it usually take for a snapshot to complete $[checkbackup.perl] module options are : --configurationfilename (string) default: config.ini current: not used as Config:IniFiles module not present --debug (number) default: 0 --help (option) default: current: 1 --pools (string) default: puddle --snapshotinterval (number) default: 300 --snapshottime (number) default: 10
I'm currently using a script at crontab to tell me when things go wrong:
#!/bin/zsh ./checkbackup.perl --pools="puddle/jolly,puddle/jolly/Pictures,puddle/jolly/Library" --snapshotinterval=7200 || say -v alex "pool snapshot for $pool is too old" ./checkbackup.perl --pools="example.com:rootpool/puddle/jolly/Pictures" --snapshotinterval=7200 || say -v alex "pool snapshot for $pool is too old"
TimeMachine backups to ZFS Volumes
I now have moved everything except for my boot partitions to ZFS. To have some backup of the root drive I'm backing that of with Apples provided TimeMachine and here is how I do it:
Create a zfs filesystem for the TimeMachine backups for several machines:
zfs create ocean/TimeMachine
Create a 100Gb sparsebundle for TimeMachine (my root is rather small, your mileage may vary):
hdiutil create -size 100g -library SPUD -fs JHFSX -type SPARSEBUNDLE -volname "tmmachinename" /Volumes/ocean/TimeMachine/tmmachinename.sparsebundle
Set up crontab to mount the sparsebundle every 20 minutes if it's not mounted yet. This is needed as TimeMachine will unmount the backup disk if it's a sparsebundle after backing up.
*/20 * * * * if [ ! -d /Volumes/tmtinkerbell ] ;then hdiutil attach /Volumes/ocean/TimeMachine/tmmachinename.sparsebundle; fi </dev/null >/dev/null 2>&1
Set up TimeMachine to use the sparsebundle:
tmutil setdestination -p /Volumes/tmmachinename