nixos/zrepl: init

[cole-h]

Motivation for this change

People have expressed interest in a zrepl module for NixOS, so I've done the bare minimum to get one up and running. The module conforms to RFC 42 which made it much easier to get the module up and running, considering there was no real "logic" to implement.

Note that I have tested none of the functionality of the service yet, and thus am opening this as a draft.

I tried to have the config be validated at build time, but that doesn't work if you use a config that requires access to certs (the configcheck subcommand actually tries to read the file).

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

---

cc potentially interested parties: @grahamc @ElvishJerricco @infinisil

Example config translated from the snap example:

{
    services.zrepl = {
        enable = true;
        settings = {
            jobs = [
                {
                    name = "snapjob";
                    type = "snap";
                    filesystems = {
                        "tank/frequently_changed<" = true;
                    };

                    snapshotting = {
                        type = "periodic";
                        interval = "2m";
                        prefix = "zrepl_snapjob_";
                    };

                    pruning = {
                        keep = [
                            { type = "last_n"; count = 60; }
                        ];
                    };
                }
            ];
        };
    };
}

[cole-h]

Depends on #113540 for the package.

[cole-h]

Ran this on a test dataset and it seems to work fine.

$ zfs list tank/test -t snapshot -r
tank/test@zrepl_snapjob_20210221_014828_000       148K      -      196K  -
tank/test@zrepl_snapjob_20210221_020328_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_021828_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_023328_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_024828_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_030328_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_031828_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_033328_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_034512_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_035012_000         0B      -      236K  -
tank/test@zrepl_snapjob_20210221_035512_000         0B      -      236K  -
tank/test/dir@zrepl_snapjob_20210221_015328_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_021828_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_023328_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_024828_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_030328_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_031828_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_033328_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_034512_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_035012_000     0B      -      196K  -
tank/test/dir@zrepl_snapjob_20210221_035512_000     0B      -      196K  -

with config:

{ lib, ... }:
{
  services.zrepl = {
    enable = true;
    settings = {
      global = {
        logging = [{
          type = "syslog";
          level = "info";
          format = "human";
        }];
      };

      jobs = [
        {
          name = "snapjob";
          type = "snap";
          filesystems = {
            "tank/test<" = true;
          };

          snapshotting = {
            type = "periodic";
            interval = "5m";
            prefix = "zrepl_snapjob_";
          };

          pruning = {
            keep = [
              {
                type = "grid";
                regex = "^zrepl_.*";
                grid = lib.concatStringsSep " | " [
                  "3x5m" # 3 buckets of 1 snapshot every 5 minutes (15 minutes total)
                  "16x15m" # 16 buckets of 1 snapshot every 15 minutes (4 hours total)
                  "6x4h" # 6 buckets of 1 snapshot every 4 hours (1 day total)
                  "7x1d" # 7 buckets of 1 snapshot every 1 day (1 week total)
                  "4x1w" # 4 buckets of 1 snapshot every 1 week (~1 month total)
                  "12x30d" # 12 buckets of 1 snapshot every 30 days (~1 year total)
                  # (roughly) translated from znapzend config:
                  # plan = "15min=>5min,4h=>15min,1d=>4h,1w=>1d,1m=>1w,1y=>1m";
                ];
              }
            ];
          };
        }
      ];
    };
  };
}

One thing to keep in mind is that zrepl doesn't support month or year as an interval, so you must use something like 30d or 365d (or whatever conversion you prefer) in those cases.

[cole-h]

@ofborg eval

[tomberek]

As a test; added this service to a copy of nixos/tests/zfs.nix and started without issues, but I did not test zrepl-specific functionality.

[grahamc]

lgtm, thanks! I'm looking forward to using this on haumea!

[Baughn]

@cole-h

I never got around to finishing it, but if you want to build something with structured config options, here's a start: https://github.com/Baughn/machine-config/tree/master/zrepl

[Maxwell-lt]

@Baughn I'm guessing you meant to link this file: https://github.com/Baughn/machine-config/blob/master/modules/zrepl.nix

I'm also using Baughn's zrepl module, though I added the ability to specify the name of a push job separately from the server CN, which is required if you want multiple push jobs to the same sink, to support multiple root pools (in my case, a fast SSD pool and a "slow" HDD pool)
https://github.com/Maxwell-lt/machine-configuration/blob/master/services/zrepl.nix

[cole-h]

tbh, I'm quite happy with this "unstructured" config version -- it's easily extendable if they ever add or remove things from their configuration format without needing to touch the module itself. But if you feel like structuring / typing this, I'll gladly review a PR :)

(I just don't feel the need to do submit a PR to that effect, myself.)