Packages are still built against zfsStable when boot.zfs.enableUnstable

[Atemu]

Describe the bug
A clear and concise description of what the bug is.

When you set boot.zfs.enableUnstable, the zfsUnstable module and userspace packages get installed. Packages that compile against ZFS like Docker and libvirt are still built against zfsStable however.

I am not sure if this would cause issues but it certainly sounds like something that could.

Expected behavior
A clear and concise description of what you expected to happen.

All ZFS-dependant packages should be built with zfs = zfsUnstable; when it is enabled.

Additional context
Add any other context about the problem here.

Came up while investigating #109001

Notify maintainers

@Mic92 @globin @wizeman @hmenke @jcumming @fpletz

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[hmenke]

I've noticed this as well and there is no solution unless nixpkgs adopts USE flags (cf. #56227) or the new module system for packages (cf. https://cfp.nixcon.org/nixcon2020/talk/K89WJY) is finalized. The same issue applies to

{
  nix.package = pkgs.nixFlakes;
}

This will also not be respected if a package explicitly has nix as a dependency. One possible solution is the steamroller approach to just globally override zfs with zfsUnstable in an overlay

{
  nixpkgs.overlays = [ (final: prev: {
    zfs = final.zfsUnstable;
  }) ];
}

This will of course start rebuilding everything that depends on zfs.

[hmenke]

But I agree that this is a big problem, especially for ZFS where you might accidentally corrupt your pool by acting with outdated user-space tool on an upgraded pool.

[hmenke]

Ideally ZFS tooling would to the same thing as modules using systemd and rely on the fact that the ZFS userspace utility is located at /run/current-system/sw/bin/zfs and not use zfs as an explicit dependency in the first place. Unfortunately, I don't know whether that would work for Docker which presumably links against ZFS as a dynamic library.

[hmenke]

Oh boy, GRUB depends on zfs, so you are lucky that you apparently do not boot from ZFS.

[NeQuissimus]

We cannot/should not make the module add a full overlay, correct?

[Mic92]

Ideally ZFS tooling would to the same thing as modules using systemd and rely on the fact that the ZFS userspace utility is located at /run/current-system/sw/bin/zfs and not use zfs as an explicit dependency in the first place. Unfortunately, I don't know whether that would work for Docker which presumably links against ZFS as a dynamic library.

I implemented this bit. It uses the executable at runtime.

[Atemu]

I don't think it's that critical. I imagine most programs that have ZFS in their buildinputs only use features that have been stable in ZFS for ages and I'd expect ZFS itself to have sanity checks for at least the current two stable releases.

The worst thing that could happen would probably be the program itself not working correctly: GRUB not beeing able to read a pool because of incompatible features for example.

Not ideal of course but I don't think data loss is a possibility here.

I think the overlay would be a good solution. This way we can be sure the change propagates through to everything that uses it in a pure and well-defined manner.
If you explicitly need the stable ZFS somewhere, you can simply reference it directly via zfsStable but the meaning of the zfs package as a whole would simply be decided by boot.zfs.enableUnstable. Kinda intuitive actually.

Would it be feasible to make hydra build every ZFS dependent package with zfsUnstable too? That way we could prevent local builds.
Although I'm not sure how bad those would be anyways, how many ZFS-dependent packages do we have, how many of those would not have to be built locally usually and how long would that take?

[rbrewer123]

I'd really like to get this tightened up so there is no question what ZFS versions are being used by different tools on my system. The overlay sounds like a good solution.

I'm trying to help track down a pretty severe bug with OpenZFS 2.0 (openzfs/zfs#10697) as I am running with zfsUnstable in NixOS 20.09. Having additional uncertainty about ZFS versions feels like the exact thing I'm trying to sidestep by running NixOS in the first place.

[stale]

I marked this as stale due to inactivity. → More info

[Atemu]

Had a quick look at how much an overlay would cost us in rebuilds and it doesn't look like much. The only major dependent in my system closure is libvirt which takes a few minutes to build at worst.

All packages that depend on zfs

• bareos
• ceph
• ceph
• ceph-client
• ceph-csi
• collectd
• collectd-data
• diffoscope
• docker-machine-kvm
• docker-machine-kvm2
• easysnap-unstable
• gnome-boxes
• libguestfs
• libguestfs
• libvirt
• libvirt-glib
• libvmi
• minikube
• minishift
• nixops-with-plugins
• ocaml-libvirt
• partitionmanager
• perl5.32.1-Sys-Virt
• perl5.34.0-Sys-Virt
• python3.8-guestfs
• python3.8-libvirt
• python3.9-guestfs
• python3.9-libvirt
• qemu
• ruby2.6.8-ruby-libvirt
• ruby2.7.4-ruby-libvirt
• ruby3.0.2-ruby-libvirt
• samba
• sanoid
• terraform-provider-libvirt
• terraform-provider-libvirt
• terraform-with-plugins
• vagrant
• virt-manager
• virt-manager
• virt-manager-qt
• virt-manager-qt
• virt-top
• virt-viewer
• virt-viewer
• virtlyst
• zfs
• zfstools

I think that makes it a straightforward and easy solution without many downsides.

[stale]

I marked this as stale due to inactivity. → More info