-
-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nixos/qemu-vm: use persistent block device names #236656
nixos/qemu-vm: use persistent block device names #236656
Conversation
This is beautiful. :) |
87c80f3
to
d711d69
Compare
Could this solve the issue trying to be solved in #233847 ? |
Only partially, here's the issue is related to Nix closures I believe |
I'm not familiar with this part and I can't do a deep dive now, but the idea seems good and I don't see any problems besides breaking tests that depend on these details, which you've already identified and I think is worth breaking. |
Once we are done with directBoot, I can push your branch to our Hydra jobset. |
Pushing 532ea30 as a baseline evaluation. |
Hello hello, I bring the failures: https://hydra.nixos.org/eval/1796336#tabs-now-fail — :D |
d711d69
to
2c8fce2
Compare
I think I fixed all deterministic failures. @RaitoBezarius can you push another eval? |
2c8fce2
to
ee60d30
Compare
Pushed ee60d30 |
Looks like only unrelated failures are left: https://hydra.nixos.org/eval/1796429?compare=-184800&full=0 |
nods |
Please write a release note for this, maybe introduce a nixpkgs internal change section in 23.11 based on what we have done for 23.05. |
This change removes the bespoke logic around identifying block devices. Instead of trying to find the right device by iterating over `qemu.drives` and guessing the right partition number (e.g. /dev/vda{1,2}), devices are now identified by persistent names provided by udev in /dev/disk/by-*. Before this change, the root device was formatted on demand in the initrd. However, this makes it impossible to use filesystem identifiers to identify devices. Now, the formatting step is performed before the VM is started. Because some tests, however, rely on this behaviour, a utility function to replace this behaviour in added in /nixos/tests/common/auto-format-root-device.nix. Devices that contain neither a partition table nor a filesystem are identified by their hardware serial number which is injecetd via QEMU (and is thus persistent and predictable). PCI paths are not a reliably way to identify devices because their availability and numbering depends on the QEMU machine type. This change makes the module more robust against changes in QEMU and the kernel (non-persistent device naming) and by decoupling abstractions (i.e. rootDevice, bootPartition, and bootLoaderDevice) enables further improvement down the line.
ee60d30
to
0bdba6c
Compare
Done |
This broke |
Ah, after reading the commit more, maybe it's as simple as turning off |
Why can't mkfs.ext4 run on Darwin? |
Hmm, I wasn't expecting The actual observed issue is that the VM run script references a Linux |
Unfortunately it is not that simple:
I don't really have any idea where to start debugging it. Homebrew has a package that apparently works, so it should be possible to fix. I'll try to take a look tomorrow. |
I opened #238596 which should at least use the right |
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/darwin-builder-on-an-m1-mac-problems-with-mkfs-ext4/29523/3 |
Apparently e2fsprogs built for Darwin until the recent upgrade, so bisecting might be an option? |
I tracked it down to #238791 which will be fixed with the upcoming Darwin stdenv rework; sorry for not linking back. |
NixOS/nixpkgs#236656 changed NixOS's qemu-vm disk-finding mechanism to use filesystem labels. squashfs doesn't support filesystem labels (see plougher/squashfs-tools#59 ). So we can't use squashfs anymore. :( The simple test's nix store image is 240M as squashfs and 1.3G as ext4.
This broke |
My bad, I need to remove the old |
Description of changes
This change removes the bespoke logic around identifying block devices. Instead of trying to find the right device by iterating over
qemu.drives
and guessing the right partition number (e.g. /dev/vda{1,2}), devices are now identified by persistent names provided by udev in /dev/disk/by-*.Before this change, the root device was formatted on demand in the initrd. However, this makes it impossible to use filesystem identifiers to identify devices. Now, the formatting step is performed before the VM is started. Because some tests, however, rely on this behaviour, a utility function to replace this behaviour in added in /nixos/tests/common/auto-format-root-device.nix.
Devices that contain neither a partition table nor a filesystem are identified by their hardware serial number which is injecetd via QEMU (and is thus persistent and predictable). PCI paths are not a reliably way to identify devices because their availability and numbering depends on the QEMU machine type.
This change makes the module more robust against changes in QEMU and the kernel (non-persistent device naming) and by decoupling abstractions (i.e. rootDevice, bootPartition, and bootLoaderDevice) enables further improvement down the line.
For all the tests to succeed, depends on #234030.Things done
sandbox = true
set innix.conf
? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"
. Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/
)