PR Merger

CHANGELIST

@@ -1,8 +1,8 @@
 2.1.0  [overall]   documentation updates, small fixes (@HavardLine, @croadfeldt, @jimsalterjrs, @jim-perkins, @kr4z33, @phreaker0)
        [syncoid]   do not require user to be specified for syncoid (@aerusso)
        [syncoid]   implemented option for keeping sync snaps (@phreaker0)
-       [syncoid]   use sudo if neccessary for checking pool capabilities regarding resumeable send (@phreaker0)
-       [syncoid]   catch another case were the resume state isn't availabe anymore (@phreaker0)
+       [syncoid]   use sudo if necessary for checking pool capabilities regarding resumable send (@phreaker0)
+       [syncoid]   catch another case were the resume state isn't available anymore (@phreaker0)
        [syncoid]   check for an invalid argument combination (@phreaker0)
        [syncoid]   fix iszfsbusy check for similar dataset names (@phreaker0)
        [syncoid]   append timezone offset to the syncoid snapshot name to fix DST collisions (@phreaker0)
@@ -29,7 +29,7 @@
 2.0.2  [overall]   documentation updates, new dependencies, small fixes, more warnings (@benyanke, @matveevandrey, @RulerOf, @klemens-u, @johnramsden, @danielewood, @g-a-c, @hartzell, @fryfrog, @phreaker0)
        [sanoid]    changed and simplified DST handling (@shodanshok)
        [syncoid]   reset partially resume state automatically (@phreaker0)
-       [syncoid]   handle some zfs erros automatically by parsing the stderr outputs (@phreaker0)
+       [syncoid]   handle some zfs errors automatically by parsing the stderr outputs (@phreaker0)
        [syncoid]   fixed ordering of snapshots with the same creation timestamp (@phreaker0)
        [syncoid]   don't use hardcoded paths (@phreaker0)
        [syncoid]   fix for special setup with listsnapshots=on (@phreaker0)
@@ -84,7 +84,7 @@
        [sanoid]    implemented monitor-capacity flag for checking zpool capacity limits (@phreaker0)
        [syncoid]   Added support for ZStandard compression.(@danielewood)
        [syncoid]   implemented support for excluding datasets from replication with regular expressions (@phreaker0)
-       [syncoid]   correctly parse zfs column output, fixes resumeable send with datasets containing spaces (@phreaker0)
+       [syncoid]   correctly parse zfs column output, fixes resumable send with datasets containing spaces (@phreaker0)
        [syncoid]   added option for using extra identification in the snapshot name for replication to multiple targets (@phreaker0)
        [syncoid]   added option for skipping the parent dataset in recursive replication (@phreaker0)
        [syncoid]   typos (@UnlawfulMonad, @jsavikko, @phreaker0)
@@ -118,12 +118,12 @@
        replicating to target/parent/child2.  This could still use some cleanup TBH; syncoid SHOULD exit 3
        if any of these errors happen (to assist detection of errors in scripting) but now would exit 0.
 
-1.4.12 Sanoid now strips trailing whitespace in template definitions in sanoid.conf, per Github #61
+1.4.12 Sanoid now strips trailing whitespace in template definitions in sanoid.conf, per GitHub #61
 
 1.4.11 enhanced Syncoid to use zfs `guid` property rather than `creation` property to ensure snapshots on source
        and target actually match. This immediately prevents conflicts due to timezone differences on source and target,
        and also paves the way in the future for Syncoid to find matching snapshots even after `zfs rename` on source
-       or target. Thank you Github user @mailinglists35 for the idea!
+       or target. Thank you GitHub user @mailinglists35 for the idea!
 
 1.4.10 added --compress=pigz-fast and --compress=pigz-slow. On a Xeon E3-1231v3, pigz-fast is equivalent compression
        to --compress=gzip but with compressed throughput of 75.2 MiB/s instead of 18.1 MiB/s. pigz-slow is around 5%
@@ -241,4 +241,4 @@
 
 1.0.1 ported slightly modified iszfsbusy sub from syncoid to sanoid (to keep from thinning snapshots during replications)
 
-1.0.0 initial commit to Github
+1.0.0 initial commit to GitHub

INSTALL.md

@@ -22,44 +22,52 @@ Install prerequisite software:
 
 ```bash
 
-apt install debhelper libcapture-tiny-perl libconfig-inifiles-perl pv lzop mbuffer build-essential
+apt install debhelper libcapture-tiny-perl libconfig-inifiles-perl pv lzop mbuffer build-essential git
 
 ```
 
 Clone this repo, build the debian package and install it (alternatively you can skip the package and do it manually like described below for CentOS):
 
 ```bash
-# Download the repo as root to avoid changing permissions later
-sudo git clone https://github.com/jimsalterjrs/sanoid.git
+git clone https://github.com/jimsalterjrs/sanoid.git
 cd sanoid
 # checkout latest stable release or stay on master for bleeding edge stuff (but expect bugs!)
 git checkout $(git tag | grep "^v" | tail -n 1)
 ln -s packages/debian .
 dpkg-buildpackage -uc -us
-apt install ../sanoid_*_all.deb
+sudo apt install ../sanoid_*_all.deb
 ```
 
 Enable sanoid timer:
 ```bash
 # enable and start the sanoid timer
-sudo systemctl enable sanoid.timer
-sudo systemctl start sanoid.timer
+sudo systemctl enable --now sanoid.timer
 ```
 
-## CentOS
+## CentOS/RHEL
 
 Install prerequisite software:
 
 ```bash
-# Install and enable epel if we don't already have it, and git too
+# Install and enable EPEL if we don't already have it, and git too:
+# (Note that on RHEL we cannot enable EPEL with the epel-release
+# package, so you should follow the instructions on the main EPEL site.)
 sudo yum install -y epel-release git
+# On CentOS, we also need to enable the PowerTools repo:
+sudo yum config-manager --set-enabled powertools
+# For Centos 8 you need to enable the PowerTools repo to make all the needed Perl modules available (Recommended)
+sudo dnf config-manager --set-enabled powertools
+# On RHEL, instead of PowerTools, we need to enable the CodeReady Builder repo:
+sudo subscription-manager repos --enable=codeready-builder-for-rhel-8-x86_64-rpms
 # Install the packages that Sanoid depends on:
-sudo yum install -y perl-Config-IniFiles perl-Data-Dumper perl-Capture-Tiny lzop mbuffer mhash pv
-# if the perl dependencies can't be found in the configured repositories you can install them from CPAN manually:
+sudo yum install -y perl-Config-IniFiles perl-Data-Dumper perl-Capture-Tiny perl-Getopt-Long lzop mbuffer mhash pv
+# The repositories above should contain all the relevant Perl modules, but if you
+# still cannot find them then you can install them from CPAN manually:
 sudo dnf install perl-CPAN perl-CPAN
-cpan # answer the questions and past the following lines
+cpan # answer the questions and paste the following lines:
 # install Capture::Tiny
 # install Config::IniFiles
+# install Getopt::Long
 ```
 
 Clone this repo, then put the executables and config files into the appropriate directories:
@@ -142,8 +150,7 @@ sudo systemctl daemon-reload
 # Enable sanoid-prune.service to allow it to be triggered by sanoid.service
 sudo systemctl enable sanoid-prune.service
 # Enable and start the Sanoid timer
-sudo systemctl enable sanoid.timer
-sudo systemctl start sanoid.timer
+sudo systemctl enable --now sanoid.timer
 ```
 
 Now, proceed to configure [**Sanoid**](#configuration)
@@ -171,6 +178,51 @@ For Alpine Linux this can be done with:
 
 `apk --no-cache add procps`
 
+## MacOS
+
+Install prerequisite software:
+
+```
+perl -MCPAN -e install Config::IniFiles
+```
+
+The crontab can be used as on a normal unix. To use launchd instead, this example config file can be use can be used. Modify it for your needs. In particular, adjust the sanoid path.
+It will start sanoid once per hour, at minute 51. Missed invocations due to standby will be merged into a single invocation at the next wakeup.
+
+```bash
+cat << "EOF" | sudo tee /Library/LaunchDaemons/net.openoid.Sanoid.plist
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>Label</key>
+	<string>net.openoid.Sanoid</string>
+	<key>ProgramArguments</key>
+	<array>
+		<string>/usr/local/sanoid/sanoid</string>
+		<string>--cron</string>
+	</array>
+	<key>EnvironmentVariables</key>
+	<dict>
+		<key>TZ</key>
+		<string>UTC</string>
+		<key>PATH</key>
+		<string>/usr/local/zfs/bin:$PATH:/usr/local/bin</string>
+	</dict>
+	<key>StartCalendarInterval</key>
+	<array>
+		<dict>
+			<key>Minute</key>
+			<integer>51</integer>
+		</dict>
+	</array>
+</dict>
+</plist>
+EOF
+
+sudo launchctl load /Library/LaunchDaemons/net.openoid.Sanoid.plist
+```
+
 ## Other OSes
 
 **Sanoid** depends on the Perl module Config::IniFiles and will not operate without it. Config::IniFiles may be installed from CPAN, though the project strongly recommends using your distribution's repositories instead.
@@ -206,3 +258,12 @@ Adapt the timer interval to the lowest configured snapshot interval.
 Take a look at the files `sanoid.defaults.conf` and `sanoid.conf` for all possible configuration options.
 
 Also have a look at the README.md for a simpler suggestion for `sanoid.conf`.
+
+## Syncoid
+If you are pushing or pulling from a remote host, create a user with privileges to `ssh` as well as `sudo`. To ensure that `zfs send/receive` can execute, adjust the privileges of the user to execute `sudo` **without** a password for only the `zfs` binary (run `which zfs` to find the path of the `zfs` binary). Modify `/etc/sudoers` by running `# visudo`. Add the following line for your user.
+
+```
+...
+<user> ALL=NOPASSWD: <path of zfs binary>
+...
+```

README.md

@@ -1,15 +1,15 @@
 <p align="center"><img src="http://www.openoid.net/wp-content/themes/openoid/images/sanoid_logo.png" alt="sanoid logo" title="sanoid logo"></p>
 
-<img src="http://openoid.net/gplv3-127x51.png" width=127 height=51 align="right">Sanoid is a policy-driven snapshot management tool for ZFS filesystems.  When combined with the Linux KVM hypervisor, you can use it to make your systems <a href="http://openoid.net/transcend" target="_blank">functionally immortal</a>.  
+<img src="http://openoid.net/gplv3-127x51.png" width=127 height=51 align="right">Sanoid is a policy-driven snapshot management tool for ZFS filesystems.  When combined with the Linux KVM hypervisor, you can use it to make your systems <a href="http://openoid.net/transcend" target="_blank">functionally immortal</a>.
 
 <p align="center"><a href="https://youtu.be/ZgowLNBsu00" target="_blank"><img src="http://www.openoid.net/sanoid_video_launcher.png" alt="sanoid rollback demo" title="sanoid rollback demo"></a><br clear="all"><sup>(Real time demo: rolling back a full-scale cryptomalware infection in seconds!)</sup></p>
 
-More prosaically, you can use Sanoid to create, automatically thin, and monitor snapshots and pool health from a single eminently human-readable TOML config file at /etc/sanoid/sanoid.conf.  (Sanoid also requires a "defaults" file located at /etc/sanoid/sanoid.defaults.conf, which is not user-editable.)  A typical Sanoid system would have a single cron job but see INSTALL.md fore more details:
+More prosaically, you can use Sanoid to create, automatically thin, and monitor snapshots and pool health from a single eminently human-readable TOML config file at /etc/sanoid/sanoid.conf.  (Sanoid also requires a "defaults" file located at /etc/sanoid/sanoid.defaults.conf, which is not user-editable.)  A typical Sanoid system would have a single cron job but see INSTALL.md for more details:
 ```
 * * * * * TZ=UTC /usr/local/bin/sanoid --cron
 ```
 
-`Note`: Using UTC as timezone is recommend to prevent problems with daylight saving times
+`Note`: Using UTC as timezone is recommended to prevent problems with daylight saving times
 
 And its /etc/sanoid/sanoid.conf might look something like this:
 
@@ -95,15 +95,15 @@ For more full details on sanoid.conf settings see [Wiki page](https://github.com
 
 + --quiet
 
-	Supress non-error output.
+	Suppress non-error output.
 
 + --verbose
 
 	This prints additional information during the sanoid run.
 
 + --debug
 
-	This prints out quite alot of additional information during a sanoid run, and is normally not needed.
+	This prints out quite a lot of additional information during a sanoid run, and is normally not needed.
 
 + --readonly
 
@@ -125,7 +125,7 @@ Will be executed before the snapshot(s) of a single dataset are taken. The follo
 | -----------------  | -----------                                                                                                                                          |
 | `SANOID_SCRIPT`    | The type of script being executed, one of `pre`, `post`, or `prune`.  Allows for one script to be used for multiple tasks                            |
 | `SANOID_TARGET`    | **DEPRECATED** The dataset about to be snapshot (only the first dataset will be provided)                                                            |
-| `SANOID_TARGETS`   | Comma separated list of all datasets to be snapshoted (currently only a single dataset, multiple datasets will be possible later with atomic groups) |
+| `SANOID_TARGETS`   | Comma separated list of all datasets to be snapshotted (currently only a single dataset, multiple datasets will be possible later with atomic groups) |
 | `SANOID_SNAPNAME`  | **DEPRECATED** The name of the snapshot that will be taken (only the first name will be provided, does not include the dataset name)                 |
 | `SANOID_SNAPNAMES` | Comma separated list of all snapshot names that will be taken (does not include the dataset name)                                                    |
 | `SANOID_TYPES`     | Comma separated list of all snapshot types to be taken (yearly, monthly, weekly, daily, hourly, frequently)                                          |
@@ -232,7 +232,7 @@ syncoid root@remotehost:data/images/vm backup/images/vm
 Which would pull-replicate the filesystem from the remote host to the local system over an SSH tunnel.
 
 Syncoid supports recursive replication (replication of a dataset and all its child datasets) and uses mbuffer buffering, lzop compression, and pv progress bars if the utilities are available on the systems used.
-If ZFS supports resumeable send/receive streams on both the source and target those will be enabled as default.
+If ZFS supports resumable send/receive streams on both the source and target those will be enabled as default.
 
 As of 1.4.18, syncoid also automatically supports and enables resume of interrupted replication when both source and target support this feature.
 
@@ -286,15 +286,15 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup
 
 + --compress <compression type>
 
-	Currently accepted options: gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, lz4, xz, lzo (default) & none. If the selected compression method is unavailable on the source and destination, no compression will be used.
+	Compression method to use for network transfer. Currently accepted options: gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, lz4, xz, lzo (default) & none. If the selected compression method is unavailable on the source and destination, no compression will be used.
 
 + --source-bwlimit <limit t|g|m|k>
 
 	This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limits are desired.
 
 + --target-bwlimit <limit t|g|m|k>
 
-	This is the bandwidth limit in bytes (kbytes, mbytesm etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.
+	This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.
 
 + --no-command-checks
 
@@ -334,15 +334,15 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup
 
 + --no-resume
 
-	This argument tells syncoid to not use resumeable zfs send/receive streams.
+	This argument tells syncoid to not use resumable zfs send/receive streams.
 
 + --force-delete
 
-	Remove target datasets recursively (WARNING: this will also affect child datasets with matching snapshots/bookmarks), if there are no matching snapshots/bookmarks.
+	Remove target datasets recursively (WARNING: this will also affect child datasets with matching snapshots/bookmarks), if there are no matching snapshots/bookmarks. Also removes conflicting snapshots if the replication would fail because of a snapshot which has the same name between source and target but different contents.
 
 + --no-clone-handling
 
-	This argument tells syncoid to not recreate clones on the targe on initial sync and doing a normal replication instead.
+	This argument tells syncoid to not recreate clones on the target on initial sync, and do a normal replication instead.
 
 + --dumpsnaps
 
@@ -370,11 +370,11 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup
 
 + --quiet
 
-	Supress non-error output.
+	Suppress non-error output.
 
 + --debug
 
-	This prints out quite alot of additional information during a sanoid run, and is normally not needed.
+	This prints out quite a lot of additional information during a sanoid run, and is normally not needed.
 
 + --help
 

packages/debian/changelog

@@ -3,8 +3,8 @@ sanoid (2.1.0) unstable; urgency=medium
   [overall]   documentation updates, small fixes (@HavardLine, @croadfeldt, @jimsalterjrs, @jim-perkins, @kr4z33, @phreaker0)
   [syncoid]   do not require user to be specified for syncoid (@aerusso)
   [syncoid]   implemented option for keeping sync snaps (@phreaker0)
-  [syncoid]   use sudo if neccessary for checking pool capabilities regarding resumeable send (@phreaker0)
-  [syncoid]   catch another case were the resume state isn't availabe anymore (@phreaker0)
+  [syncoid]   use sudo if necessary for checking pool capabilities regarding resumable send (@phreaker0)
+  [syncoid]   catch another case were the resume state isn't available anymore (@phreaker0)
   [syncoid]   check for an invalid argument combination (@phreaker0)
   [syncoid]   fix iszfsbusy check for similar dataset names (@phreaker0)
   [syncoid]   append timezone offset to the syncoid snapshot name to fix DST collisions (@phreaker0)
@@ -39,7 +39,7 @@ sanoid (2.0.2) unstable; urgency=medium
   [overall]   documentation updates, new dependencies, small fixes, more warnings (@benyanke, @matveevandrey, @RulerOf, @klemens-u, @johnramsden, @danielewood, @g-a-c, @hartzell, @fryfrog, @phreaker0)
   [syncoid]   changed and simplified DST handling (@shodanshok)
   [syncoid]   reset partially resume state automatically (@phreaker0)
-  [syncoid]   handle some zfs erros automatically by parsing the stderr outputs (@phreaker0)
+  [syncoid]   handle some zfs errors automatically by parsing the stderr outputs (@phreaker0)
   [syncoid]   fixed ordering of snapshots with the same creation timestamp (@phreaker0)
   [syncoid]   don't use hardcoded paths (@phreaker0)
   [syncoid]   fix for special setup with listsnapshots=on (@phreaker0)
@@ -102,7 +102,7 @@ sanoid (2.0.0) unstable; urgency=medium
   [sanoid]    implemented monitor-capacity flag for checking zpool capacity limits (@phreaker0)
   [syncoid]   Added support for ZStandard compression.(@danielewood)
   [syncoid]   implemented support for excluding datasets from replication with regular expressions (@phreaker0)
-  [syncoid]   correctly parse zfs column output, fixes resumeable send with datasets containing spaces (@phreaker0)
+  [syncoid]   correctly parse zfs column output, fixes resumable send with datasets containing spaces (@phreaker0)
   [syncoid]   added option for using extra identification in the snapshot name for replication to multiple targets (@phreaker0)
   [syncoid]   added option for skipping the parent dataset in recursive replication (@phreaker0)
   [syncoid]   typos (@UnlawfulMonad, @jsavikko, @phreaker0)