Ceph daemons: Improve systemd services

[spacekitteh]

A lot of Ceph issues I've been having with NixOS stem from the services generated by nixpkgs. Some things which I think will help massively are:

1. Use template service files, particularly for the OSD daemons. It's rather annoying at the moment to manually modify each service of a given type when you have a large number of them in your cluster; I don't want to hardcode names, and this is a perfect use-case for templated services here.
2. Create a service for activating ceph-volume OSDs. If you use ceph-volume to provision OSDs, then at each boot, you have to run ceph-volume lvm activate --all --no-systemd /after/ monitors become available, but /before/ any OSDs provisioned by ceph-volume are started. Currently, this is what I use:

systemd.services."ceph-volume-activate" = {
  requiredBy = ["ceph-osd.target"];
  requires = ["ceph-mon.target"];
  script = "ceph-volume lvm activate --all --no-systemd";
  path = [pkgs.ceph pkgs.util-linux pkgs.lvm2 pkgs.cryptsetup];
  serviceConfig = {
    Type = "oneshot";
    RemainAfterExit = "yes";
  };
};
systemd.targets."ceph-osd".requires = ["ceph-volume-activate.service" "ceph-mon.target"];

but I'm still tinkering.
3. Ensure the daemons have the proper paths. E.g. the mgr daemons should have pkgs.ceph and pkgs.smartmontools in their path (but smartctl still doesn't work on the dashboard; I'm still investigating).
4. Each daemon should have its state folder created automatically in /var/lib/ceph/, by user ceph. Currently, you have to manually create that folder.

If I think of more improvements that could be made, I'll add them here.

[deepfire]

@spacekitteh, so I'm not alone -- I've also had to use a the ceph volume activation service.

Filed a PR: #281924

[deepfire]

@spacekitteh, can't add you to reviewers, for some reason.