boot.initrd.systemd.enable on 23.05 breaks raid configurations

[gpl]

Describe the bug

I'm attempting to utilize systemd in initrd.

Steps To Reproduce

Steps to reproduce the behavior:

set boot.initrd.systemd.enable = true on a system with ext4(luks(raid1(disk)))

lsblk output:

NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
nvme2n1     259:0    0 186.3G  0 disk
└─nvme2n1p1 259:1    0 186.3G  0 part
nvme1n1     259:2    0 894.3G  0 disk
├─nvme1n1p1 259:4    0   512M  0 part
│ └─md127     9:127  0 511.9M  0 raid1 /boot
└─nvme1n1p2 259:5    0 893.8G  0 part
  └─md126     9:126  0 893.6G  0 raid1
    └─nixos 254:0    0 893.6G  0 crypt /nix/store
                                       /
nvme0n1     259:3    0 894.3G  0 disk
├─nvme0n1p1 259:6    0   512M  0 part
│ └─md127     9:127  0 511.9M  0 raid1 /boot
└─nvme0n1p2 259:7    0 893.8G  0 part
  └─md126     9:126  0 893.6G  0 raid1
    └─nixos 254:0    0 893.6G  0 crypt /nix/store
                                       /

Upon setting boot.initrd.systemd.enable = true; the system claims there are no arrays in /proc/mdstat. There are no md devices in /dev/disk/by-id but the raw block devices are present.

    initrd = {
      availableKernelModules = [ "dummy" "sha1" "sha256" "aes" "xts" "encrypted_keys" "cryptd" "dm_crypt" "dm_mod" "mlx5_core" "bonding" "8021q" "wireguard" ];
      kernelModules = [ "cryptd" "dm_crypt" "dm_mod" "vhost_vsock" "overlay" "raid1" ];
      systemd.enable = true;
      systemd.network.enable = true;
      systemd.emergencyAccess = true;
      network = {
        flushBeforeStage2 = true;
        enable = true;
        ssh = {
          enable = true;
          port = 2222;
          hostKeys = [
              "/etc/secrets/initrd/ssh_host_ed25519_key"
          ];
        };
      };
    };

Expected Behavior

system prompts to unlock md devices; instead it drops to emergency mode

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.31, NixOS, 23.05 (Stoat), 23.05.553.e7603eba51f`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - channels(root): `"nixos-23.05, nixos-unstable"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[flokli]

cc @ElvishJerricco @RaitoBezarius

[ElvishJerricco]

So this is because swraid is disabled by default with systemd initrd, but it seems I missed that option when unhiding the docs.

This is something that's still kind of in-development. We have a PR that addresses the issue using system.stateVersion to determine whether to enable these by default and add their detection to nixos-generate-config. That way existing systems don't need any change, and new systems detect this when generating hardware-configuration.nix.

It's all subject to change at the moment; after all, systemd initrd is still marked as experimental (though it's gotten pretty stable for something experimental). But what you can do for now to ensure it works the same as before is add one or both of these two lines depending on if you need mdraid and/or lvm:

boot.initrd.services.swraid.enable = true;
boot.initrd.services.lvm.enable = true;

[RaitoBezarius]

Can we provide the adequate asserts/warnings for this?

[ElvishJerricco]

@RaitoBezarius Not sure how we would detect that a warning/assertion is necessary. It's not like we can actually see the disk topology at eval time. We could make some guesses like "I see you used /dev/md0 for your root device but haven't enabled initrd swraid", but that will be pretty far from perfect, especially considering most people use a by-label or by-uuid link instead of a kernel devname like md0 that indicates what kind of thing it is.

[Atemu]

This should perhaps be documented somewhere. Though we don't document it for LVM either.

[ElvishJerricco]

@Atemu I think we just need to implement the stateVersion thing described above, so that swraid is enabled by default on old systems, and detected by nixos-generate-config for new systems

[cmm]

what's up with this? I won't be able to upgrade my NAS to 26.11 if this is still a thing, right?

[ElvishJerricco]

AFAIK it should work fine now and the issue can be closed. You just need to add boot.initrd.services.swraid.enable = true; to your config if you need it in stage 1, but even that should be handled automatically by the stateVersion and generated hardware-configuration.nix things we've had for a while now. If something isn't working then feel free to reopen this or open a new issue with details.