nixos-generate-config interacts badly with swapDevices.*.randomEncryption.enable=true

[falsifian]

Describe the bug
My swap devices are encrypted with random keys, using swapDevices.*.randomEncryption.enable = true.

When I run nixos-generate-config, I get something like this in hardware-configuration.nix:

swapDevices =
  [ { device = "/dev/disk/by-uuid/8c28d4f9-6ad2-4295-a43c-344533ef56d5"; }
    { device = "/dev/disk/by-uuid/2eccea7d-d7ed-426e-8ef1-bc96246e9256"; }
  ];

The trouble is, those uuids won't exist on the next boot (I think they're different every time), so NixOS hangs for 90 seconds during boot waiting for them to appear.

I know of three workarounds:

• Edit hardware-configuration.nix to remove them. But I'm not supposed to edit that file, according to the comment at the top.
• Use lib.mkForce in configuration.nix to override the swapDevices from hardware-configuration.nix. But this breaks zramSwap: see zramSwap.enable doesn't work with nixos-rebuild build-vm #86061
• I just discovered nixos-generate-config has a --no-filesystems option. I haven't tried it yet, but I assume it will fix my problem.

To Reproduce

1. Enable randomEncryption. For example, my configuration.nix is posted here. To properly reproduce the problem, you should leave out mkForce from the swapDevices attribute (I put that in as a workaround.)
2. After rebooting (or maybe just switching) with randomEncryption enabled, run nixos-generate-config. Observe the swapDevices entry in hardware-configuration.nix pointing to ephemeral UUIDs.
3. If you run nixos-rebuild boot and reboot, and you didn't use the mkForce workaround, then you should see the system hang for 90s on boot waiting for those old ephemeral UUIDs to appear.

Expected behavior
nixos-generate-config should not add the swap devices to hardware-configuration.nix.

(Why do filesystems and swap devices go into hardware-configuration.nix rather than configuration.nix? It seems like it's useful to have those automated the first time I install NixOS, but I can't think of a reason I'd want nixos-generate-config to change them on its own later. Having nixos-generate-config put them in configuration.nix would achieve the behaviour I want, which is to not mess with them after installation.)

Notify maintainers
Picking on two people who authered recent-ish commits in nixos-generate-config.pl:
@bjornfor
@buckley310

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

• system: "x86_64-linux"
• host os: Linux 5.4.33, NixOS, 20.03.1445.95b9c99f6d0 (Markhor)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.4
• channels(james): ""
• channels(root): "nixos-20.03.1445.95b9c99f6d0"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[buckley310]

The logic in the perl script to choose which devices get included in hardware-configuration is pretty rudimentary, it just excludes files and zram devices, and includes everything else. The only robust way I can think of to selectively exclude block devices would be for the nixos derivation to produce a list of devices that should be skipped, instead of trying to decide in the perl script.

For example, what if every swapDevices entry had a property autoGenerated which defaults to false, but all the swap entries in hardware-configuration.nix would have this set to true. On every rebuild, nix could produce a list at sw/share/nixos/manualy-configured-swap-devices which contains all the devices with autoGenerated=false, and the perl script would just skip these.

Since the UUID of randomly encrypted devices is not known at build time, but the /dev/mapper/* path is, the nix derivation could add the latter to the blacklist, and the perl script could be modified to prefer the /dev/mapper/* form over the /dev/disk/by-uuid/* form, unless anyone knows of a case where that would cause problems.

[falsifian]

@buckley310 Sounds good as far as I can tell.

I still think it would be simpler to put filesystems and swap in configuration.nix rather than hardware-configuration.nix, so that nixos-generate-config just doesn't touch that stuff after the initial install. But maybe there's a reason I'm not thinking of for having it the way it is.

[buckley310]

This took less effort than i expected. I'm working on it here https://github.com/buckley310/nixpkgs/tree/swap-tracking
Still need to test it some more...

i think the reason filesystems go in the hardware config is so that its technically possible to have everything except the hardware config shared across multiple machines, but I'm not totally sure. I don't use it that way.

[falsifian]

Thanks for working on that. I'm not currently running the NixOS installation where I encountered that problem, but will keep your WIP in mind if I boot that up again.

[stale]

I marked this as stale due to inactivity. → More info