Merge branch 'master' into master

INSTALL.md

@@ -0,0 +1,172 @@
+# Installation
+
+**Sanoid** and **Syncoid** are complementary but separate pieces of software. To install and configure them, follow the guide below for your operating system. Everything in `code blocks` should be copy-pasteable. If your OS isn't listed, a set of general instructions is at the end of the list and you can perform the process manually.
+
+<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:0 orderedList:0 -->
+
+- [Installation](#installation)
+	- [Debian/Ubuntu](#debianubuntu)
+	- [CentOS](#centos)
+	- [FreeBSD](#freebsd)
+	- [Other OSes](#other-oses)
+- [Configuration](#configuration)
+	- [Sanoid](#sanoid)
+
+<!-- /TOC -->
+
+
+## Debian/Ubuntu
+
+Install prerequisite software:
+
+```bash
+apt install libconfig-inifiles-perl pv lzop mbuffer
+```
+
+Clone this repo, build the debian package and install it (alternatively you can skip the package and do it manually like described below for CentOS):
+
+```bash
+# Download the repo as root to avoid changing permissions later
+sudo git clone https://github.com/jimsalterjrs/sanoid.git
+cd sanoid
+ln -s packages/debian .
+dpkg-buildpackage -uc -us
+apt install ../sanoid_*_all.deb
+```
+
+Enable sanoid timer:
+```bash
+# enable and start the sanoid timer
+sudo systemctl enable sanoid.timer
+sudo systemctl start sanoid.timer
+```
+
+## CentOS
+
+Install prerequisite software:
+
+```bash
+# Install and enable epel if we don't already have it, and git too
+sudo yum install -y epel-release git
+# Install the packages that Sanoid depends on:
+sudo yum install -y perl-Config-IniFiles perl-Data-Dumper lzop mbuffer mhash pv
+```
+
+Clone this repo, then put the executables and config files into the appropriate directories:
+
+```bash
+# Download the repo as root to avoid changing permissions later
+sudo git clone https://github.com/jimsalterjrs/sanoid.git
+cd sanoid
+# Install the executables
+sudo cp sanoid syncoid findoid sleepymutex /usr/local/sbin
+# Create the config directory
+sudo mkdir /etc/sanoid
+# Install default config
+sudo cp sanoid.defaults.conf /etc/sanoid
+# Create a blank config file
+sudo touch /etc/sanoid/sanoid.conf
+# Place the sample config in the conf directory for reference
+sudo cp sanoid.conf /etc/sanoid/sanoid.example.conf
+```
+
+Create a systemd service:
+
+```bash
+cat << "EOF" | sudo tee /etc/systemd/system/sanoid.service
+[Unit]
+Description=Snapshot ZFS Pool
+Requires=zfs.target
+After=zfs.target
+ConditionFileNotEmpty=/etc/sanoid/sanoid.conf
+
+[Service]
+Environment=TZ=UTC
+Type=oneshot
+ExecStart=/usr/local/sbin/sanoid --take-snapshots --verbose
+EOF
+
+cat << "EOF" | sudo tee /etc/systemd/system/sanoid-prune.service
+[Unit]
+Description=Cleanup ZFS Pool
+Requires=zfs.target
+After=zfs.target sanoid.service
+ConditionFileNotEmpty=/etc/sanoid/sanoid.conf
+
+[Service]
+Environment=TZ=UTC
+Type=oneshot
+ExecStart=/usr/local/sbin/sanoid --prune-snapshots --verbose
+
+[Install]
+WantedBy=sanoid.service
+EOF
+```
+
+And a systemd timer that will execute **Sanoid** once per quarter hour
+(Decrease the interval as suitable for configuration):
+
+```bash
+cat << "EOF" | sudo tee /etc/systemd/system/sanoid.timer
+[Unit]
+Description=Run Sanoid Every 15 Minutes
+Requires=sanoid.service
+
+[Timer]
+OnCalendar=*:0/15
+Persistent=true
+
+[Install]
+WantedBy=timers.target
+EOF
+```
+
+Reload systemd and start our timer:
+```bash
+# Tell systemd about our new service definitions
+sudo systemctl daemon-reload
+# Enable and start the Sanoid timer
+sudo systemctl enable sanoid.timer
+sudo systemctl start sanoid.timer
+```
+
+Now, proceed to configure [**Sanoid**](#configuration)
+
+## FreeBSD
+
+Install prerequisite software:
+
+```bash
+pkg install p5-Config-Inifiles pv mbuffer lzop
+```
+
+**Additional notes:**
+
+*   FreeBSD may place pv and lzop in somewhere other than /usr/bin — syncoid currently does not check path.
+
+*   Simplest path workaround is symlinks, eg `ln -s /usr/local/bin/lzop /usr/bin/lzop` or similar, as appropriate to create links in **/usr/bin** to wherever the utilities actually are on your system.
+
+*   See note about mbuffer and other things in FREEBSD.readme
+
+## Other OSes
+
+**Sanoid** depends on the Perl module Config::IniFiles and will not operate without it. Config::IniFiles may be installed from CPAN, though the project strongly recommends using your distribution's repositories instead.
+
+**Syncoid** depends on ssh, pv, gzip, lzop, and mbuffer. It can run with reduced functionality in the absence of any or all of the above. SSH is only required for remote synchronization. On newer FreeBSD and Ubuntu Xenial chacha20-poly1305@openssh.com, on other distributions arcfour crypto is the default for SSH transport since v1.4.6. Syncoid runs will fail if one of them is not available on either end of the transport.
+
+### General outline for installation
+
+1.  Install prerequisites: Perl module Config::IniFiles, ssh, pv, gzip, lzop, and mbuffer
+2.  Download the **Sanoid** repo
+3.  Create the config directory `/etc/sanoid` and put `sanoid.defaults.conf` in there, and create `sanoid.conf` in it too
+4.  Create a cron job or a systemd timer that runs `sanoid --cron` once per minute
+
+# Configuration
+
+**Sanoid** won't do anything useful unless you tell it how to handle your ZFS datasets in `/etc/sanoid/sanoid.conf`.
+
+**Syncoid** is a command line utility that doesn't require any configuration, with all of its switches set at runtime.
+
+## Sanoid
+
+Take a look at the files `sanoid.defaults.conf` and` sanoid.conf.example` for all possible configuration options. Also have a look at the README.md

README.md

@@ -184,11 +184,11 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup
 
 + --source-bwlimit <limit t|g|m|k>
 
-	This is the bandwidth limit imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limites are desired.
+	This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limits are desired. 
 
 + --target-bw-limit <limit t|g|m|k>
 
-	This is the bandwidth limit imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limites are desired.
+	This is the bandwidth limit in bytes (kbytes, mbytesm etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.
 
 + --no-command-checks
 
@@ -202,6 +202,10 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup
 
 	This argument tells syncoid to restrict itself to existing snapshots, instead of creating a semi-ephemeral syncoid snapshot at execution time. Especially useful in multi-target (A->B, A->C) replication schemes, where you might otherwise accumulate a large number of foreign syncoid snapshots.
 
++ --create-bookmark
+
+	This argument tells syncoid to create a zfs bookmark for the newest snapshot after it got replicated successfully. The bookmark name will be equal to the snapshot name. Only works in combination with the --no-sync-snap option. This can be very useful for irregular replication where the last matching snapshot on the source was already deleted but the bookmark remains so a replication is still possible. 
+
 + --no-clone-rollback
 
 	Do not rollback clones on target

sanoid

@@ -460,16 +460,30 @@ sub take_snapshots {
 					# update to most current possible datestamp
 					%datestamp = get_date();
 					# print "we should have had a $type snapshot of $path $maxage seconds ago; most recent is $newestage seconds old.\n";
-					push(@newsnaps, "$path\@autosnap_$datestamp{'sortable'}_$type");
+
+					# use zfs (atomic) recursion if specified in config
+					if ($config{$section}{'zfs_recursion'}) {
+						push(@newsnaps, "$path\@autosnap_$datestamp{'sortable'}_$type\@");
+					} else {
+						push(@newsnaps, "$path\@autosnap_$datestamp{'sortable'}_$type");
+					}
 				}
 			}
 		}
 	}
 
 	if ( (scalar(@newsnaps)) > 0) {
 		foreach my $snap ( @newsnaps ) {
-			my $dataset = (split '@', $snap)[0];
-			my $snapname = (split '@', $snap)[1];
+			my $extraMessage = "";
+			my @split = split '@', $snap, -1;
+			my $recursiveFlag = 0;
+			if (scalar(@split) == 3) {
+				$recursiveFlag = 1;
+				$extraMessage = " (zfs recursive)";
+				chop $snap;
+			}
+			my $dataset = $split[0];
+			my $snapname = $split[1];
 			my $presnapshotfailure = 0;
 			my $ret = 0;
 			if ($config{$dataset}{'pre_snapshot_script'}) {
@@ -490,17 +504,22 @@ sub take_snapshots {
 					$presnapshotfailure = 1;
 				}
 			}
-			if ($args{'verbose'}) { print "taking snapshot $snap\n"; }
+			if ($args{'verbose'}) { print "taking snapshot $snap$extraMessage\n"; }
 			if (!$args{'readonly'}) {
-				system($zfs, "snapshot", "$snap") == 0
-					or warn "CRITICAL ERROR: $zfs snapshot $snap failed, $?";
+				if ($recursiveFlag) {
+					system($zfs, "snapshot", "-r", "$snap") == 0
+						or warn "CRITICAL ERROR: $zfs snapshot -r $snap failed, $?";
+				} else {
+					system($zfs, "snapshot", "$snap") == 0
+						or warn "CRITICAL ERROR: $zfs snapshot $snap failed, $?";
+				}
 			}
 			if ($config{$dataset}{'post_snapshot_script'}) {
 				if (!$presnapshotfailure or $config{$dataset}{'force_post_snapshot_script'}) {
 					$ENV{'SANOID_TARGET'} = $dataset;
 					$ENV{'SANOID_SNAPNAME'} = $snapname;
 					if ($args{'verbose'}) { print "executing post_snapshot_script '".$config{$dataset}{'post_snapshot_script'}."' on dataset '$dataset'\n"; }
-	
+
 					if (!$args{'readonly'}) {
 						runscript('post_snapshot_script',$dataset);
 					}
@@ -713,6 +732,8 @@ sub init {
 
 	# we'll use these later to normalize potentially true and false values on any toggle keys
 	my @toggles = ('autosnap','autoprune','monitor_dont_warn','monitor_dont_crit','monitor','recursive','process_children_only','skip_children','no_inconsistent_snapshot','force_post_snapshot_script');
+	# recursive is defined as toggle but can also have the special value "zfs", it is kept to be backward compatible
+
 	my @istrue=(1,"true","True","TRUE","yes","Yes","YES","on","On","ON");
 	my @isfalse=(0,"false","False","FALSE","no","No","NO","off","Off","OFF");
 
@@ -808,26 +829,49 @@ sub init {
 		}
 
 		# how 'bout some recursion? =)
+		if ($config{$section}{'zfs_recursion'} && $config{$section}{'zfs_recursion'} == 1 && $config{$section}{'autosnap'} == 1) {
+			warn "ignored autosnap configuration for '$section' because it's part of a zfs recursion.\n";
+			$config{$section}{'autosnap'} = 0;
+		}
+
 		my $recursive = $ini{$section}{'recursive'} && grep( /^$ini{$section}{'recursive'}$/, @istrue );
+		my $zfsRecursive = $ini{$section}{'recursive'} && $ini{$section}{'recursive'} =~ /zfs/i;
 		my $skipChildren = $ini{$section}{'skip_children'} && grep( /^$ini{$section}{'skip_children'}$/, @istrue );
 		my @datasets;
-		if ($recursive || $skipChildren) {
+		if ($zfsRecursive || $recursive || $skipChildren) {
+			if ($zfsRecursive) {
+				$config{$section}{'zfs_recursion'} = 1;
+			}
+
 			@datasets = getchilddatasets($config{$section}{'path'});
 			DATASETS: foreach my $dataset(@datasets) {
 				chomp $dataset;
 
-				if ($skipChildren) {
-					if ($args{'debug'}) { print "DEBUG: ignoring $dataset.\n"; }
-					delete $config{$dataset};
-					next DATASETS;
-				}
+				if ($zfsRecursive) {
+					# don't try to take the snapshot ourself, recursive zfs snapshot will take care of that
+					$config{$dataset}{'autosnap'} = 0;
+
+					foreach my $key (keys %{$config{$section}} ) {
+						if (! ($key =~ /template|recursive|children_only|autosnap/)) {
+							if ($args{'debug'}) { print "DEBUG: recursively setting $key from $section to $dataset.\n"; }
+							$config{$dataset}{$key} = $config{$section}{$key};
+						}
+					}
+				} else {
+					if ($skipChildren) {
+						if ($args{'debug'}) { print "DEBUG: ignoring $dataset.\n"; }
+						delete $config{$dataset};
+						next DATASETS;
+					}
 
-				foreach my $key (keys %{$config{$section}} ) {
-					if (! ($key =~ /template|recursive|children_only/)) {
-						if ($args{'debug'}) { print "DEBUG: recursively setting $key from $section to $dataset.\n"; }
-						$config{$dataset}{$key} = $config{$section}{$key};
+					foreach my $key (keys %{$config{$section}} ) {
+						if (! ($key =~ /template|recursive|children_only/)) {
+							if ($args{'debug'}) { print "DEBUG: recursively setting $key from $section to $dataset.\n"; }
+							$config{$dataset}{$key} = $config{$section}{$key};
+						}
 					}
 				}
+
 				$config{$dataset}{'path'} = $dataset;
 				$config{$dataset}{'initialized'} = 1;
 			}
@@ -1380,7 +1424,7 @@ sub getchilddatasets {
 	my $fs = shift;
 	my $mysudocmd = '';
 
-	my $getchildrencmd = "$mysudocmd $zfs list -o name -Hr $fs |";
+	my $getchildrencmd = "$mysudocmd $zfs list -o name -t filesystem,volume -Hr $fs |";
 	 if ($args{'debug'}) { print "DEBUG: getting list of child datasets on $fs using $getchildrencmd...\n"; }
 	open FH, $getchildrencmd;
 	my @children = <FH>;