nixos/lib/make-ext4-fs: Fix: `resize2fs -M' can leave insufficient slack #125121
Conversation
The root filesystem resizing step, `resize2fs -M', does not provide any
control over the amount of slack left in the result. It can produce an
arbitrarily tight fit, depending on how well the payload aligns with
ext4 data structures.
This is problematic, as NixOS must create a few files and directories
during its first boot, before the root is enlarged to match the size of
the containing SD card.
An overly tight fit can cause failures in the first stage:
mkdir: can't create directory '/mnt-root/proc': No space left on device
or in the second stage:
install: cannot create directory '/var': No space left on device
A previous version of `make-ext4-fs' (before PR NixOS#79368) was explicitly
"reserving" 16 MiB of free space in the final filesystem. Manually
calculating the size of an ext4 filesystem is a perilous endeavor,
however, and the method it employed was apparently unreliable.
Reverting is consequently not a good option.
A solution would be to create some sort of "balloon" occupying inodes
and blocks in the image prior to invoking `resize2fs -M', and to remove
these temporary files/directories before the compression step.
This changeset takes the simpler approach of simply dropping the
resizing step.
Note that this does *not* result in a larger image in general, as the
current procedure does not truncate the `.img' file anyway. In fact, it
has been observed to yield *smaller* compressed images---probably
because of some "noise" left after resizing. E.g., before-vs-after:
-r--r--r-- 2 root root 607M 1. Jan 1970 nixos-sd-image-21.11pre-git-x86_64-linux.img.zst
-r--r--r-- 2 root root 606M 1. Jan 1970 nixos-sd-image-21.11pre-git-x86_64-linux.img.zst
|
How big is the uncompressed ext4 image before/after the change? Though this change is probably fine either way since it does not change the API of the image builder. In practice we really need to take the time to do the unification work of all the various bespoke image building code into a common interface. |
The uncompressed As for the … which is not very representative. |
Right, so my main concern here ends up being a no-op. (Though as I write this I realize it wasn't much of a real concern that really mattered...) |
|
There is literally no reason not to merge this AFAICT. Given that the image is already as big as it gets, so it's not like it would stop fitting on any storage media. I guess this might not actually solve some underlying issue where there is not enough slack space in the filesystem. In that instance I guess adding more slack space outright would be fine as long as the compressed image stays in the same size ballpark. |
|
Successfully created backport PR #125159 for |
|
@samueldr, @Mic92: Great; thanks! Samuel, given your conclusions, and the fact that I agree with them, I ended up focusing on other things than that ext4 filesystem today. I am still planning to experiment a bit with the SD image creation scripts, and will keep you posted as soon as I do. Cheers, -D |
|
Hi @samueldr, I had a closer look into this, based on the image on which we observed the issue. For some reason, One thing I had missed is that
There isn't much of a difference in the size of the compressed output, so this shouldn't impact Hydra. It might occasionally cause an issue if somebody tries to dump the image on a small-ish SD card, however. For the filesystem itself, we have:
As with the
(I don't have a clear explanation for the difference in compressed image size despite the fact that You wrote this:
I agree, and minimizing the size of the filesystem (while explicitly reserving some slack, to avoid the issue fixed by this PR) should ideally be part of the checklist for such an effort. Do we have a document and/or ticket I should watch/contribute to? |
Not yet |
Motivation for this change
The root filesystem resizing step,
resize2fs -M, does not provide any control over the amount of slack left in the result. It can produce an arbitrarily tight fit, depending on how well the payload aligns with ext4 data structures.This is problematic, as NixOS must create a few files and directories during its first boot, before the root is enlarged to match the size of the containing SD card.
An overly tight fit can cause failures in the first stage:
or in the second stage:
A previous version of
make-ext4-fs(before PR #79368) was explicitly "reserving" 16 MiB of free space in the final filesystem. Manually calculating the size of an ext4 filesystem is a perilous endeavor, however, and the method it employed was apparently unreliable.Reverting is consequently not a good option.
A solution would be to create some sort of "balloon" occupying inodes and blocks in the image prior to invoking
resize2fs -M, and to remove these temporary files/directories before the compression step.This changeset takes the simpler approach of simply dropping the resizing step.
Note that this does not result in a larger image in general, as the current procedure does not truncate the
.imgfile anyway. In fact, it has been observed to yield smaller compressed images—probably because of some "noise" left after resizing. E.g., before-vs-after:Things done
sandboxinnix.confon non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"./result/bin/)