boot.initrd.secrets is undocumented and appears broken

[mdorman]

Issue description

c06d795 introduced boot.initrd.secrets, in place of the prior boot.loader.grub.extraInitrd.

While I sincerely appreciate trying to handle this directly, it has four problems that I can see:

0. It arbitrarily breaks working systems
   Not even a deprecation period, just suddenly I can't rebuild.
1. No documentation!
   This seems like it should be a baseline requirement for a feature like this, but instead we removed a documented option in favor of one with no documentation. As a user, I'm extremely disappointed by this.
2. None of the seemingly obvious values worked
   Trying what seem to me the intuitive options didn't work---you can't just give it a file, and you can't just give it an array of files. I read the code, and it appears you must give it an attribute set? I can guess that my keyfile is supposed to be one or the other, but what's the other thing supposed to be?
3. If you read the code and at least guess what is supposed to happen, it doesn't appear to work because it makes assumptions that don't appear to hold on all systems.
   It appears to be assuming things about paths existing.

It seems like this could have stood more review. CC: @joachifm as committer, and @lopsided98 as author.

Steps to reproduce

Add boot.initrd.secrets = { "" = "/luks-root.key"; }; to your /etc/nixos/configuration.nix. Try to rebuild your system, and you'll see this error:

Error getting name to temp file from template /boot/kernels/initrd-secrets.XXXXXXXX: Parent directory (/boot/kernels/) does not exist at /nix/store/rhaqrbjcx5mzd7zq2819x8bm83crwl35-install-grub.pl line 360.

Technical details

Please run nix-shell -p nix-info --run "nix-info -m" and paste the
results.

• system: "x86_64-linux"
• host os: Linux 4.17.0, NixOS, 18.09.git.eddb6f9 (Jellyfish)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.0
• channels(root): ""
• channels(mdorman): ""
• nixpkgs: /var/nixup/nixpkgs

[lopsided98]

When I submitted my PR, I assumed it was ok to make a backwards incompatible to master, and that users running NixOS unstable would be willing to accept such changes.

I did not create the boot.initrd.secrets option, I simply added support for it to grub. It is documented here: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-1.nix#L476

I assume you have boot.loader.grub.copyKernels=false. I forgot to account for the fact that /boot/kernels might not exist if that option was set. I will submit a PR to fix this soon.

@joachifm Feel free to revert this until I can fix the issue.

[joachifm]

My bad. I've reverted it for now. However, I don't think it's well advised to run from master nor expect that it will always be in good shape, but I'll admit I should have looked closer, sorry.

[mdorman]

@joachifm Well I run master because I hope I can help stabilize stuff, if only by giving feedback when stuff breaks.

@lopsided98 I was surprised by the straight-up removal of the existing option because ISTM that it's been in an actual release (18.03), and thus ought to see a deprecation period---but I didn't check.

And I should apologize: you're right, I do see that the option is documented; for some reason the manpage for configuration.nix doesn't have it listed on my system. Weird. With the pointer, at least, it starts to make sense.

Finally, you're correct, I have /boot on the same drive as /, so copyKernels isn't automatically turned on, nor have I specifically set it to on.

Just to be clear: I would actually like this to go in, and am willing to play guinea pig on it a bit if that helps (though I'll be slow and laggy to respond, so I might not be the ideal testbed).

[joachifm]

@mdorman sure :) I just got the impression you were annoyed that this had broken a deployment of yours.

[Chiiruno]

boot.initrd.secrets works fine for me.
https://github.com/Chiiruno/configuration/blob/master/etc/nixos/boot.nix#L28

[lopsided98]

I have a rebased version of the PR that should fix the copyKernels issue. @mdorman If you want to test it and it works for you I'll make a new PR for it.

It still isn't backwards compatible, but the change is documented in the changelog.

[Ekleog]

(triage) @mdorman, do you want to test the new version of the PR? :)

[mdorman]

I can't get this to work. I can verify that the file gets copied into the initrd under /nix/store/<id>-extra-utils/secrets/, and I've made sure that the values in configure.nix all match up, but it never finds the keyfile.

[lopsided98]

I'm not sure I understand your problem. When is your error occurring - at boot or during activation? What are the source and destination locations of your secrets? Could you post the relevant section of your config?

[emilazy]

Any progress on un-reverting this? systemd-boot has supported initrd secrets for a while now and other NixOS modules use them; I was surprised that GRUB doesn't in #84976.

cc @joachifm

[Kloenk]

I also would like it if this could be un-reverted, because without some secrets options for grub which are executed at Activation time, I cannot build my initrd ssh. This is due the pure mode of flakes, where I would have to provide my ssh host keys as an flake, or load them after build time.

[lopsided98]

I created #85418

[spinus]

@mdorman can you check whether #85418 address your ask?

@grahamc @fpletz - that looks quite serious issue if disk encryption keys end up in both nix store and wide opened boot partition, should this be backported?

[mdorman]

I apologize, I thought this got closed. Yes, I found the change that was merged a while ago to provide sufficient documentation that I was able to convert over to using it without a problem.

I appreciate everyone's patience in getting things to that point.