Refactor the 99squash module #1088
Conversation
github-actions
bot
added
modules
There was a problem hiding this comment.
Given the scope of this change where you seem to be making the assumption that no one is using the squash module unless she/he is using buxybox as well. which I do believe is not the case here, it's probably best if these changes are introduced in a separated module ( squash-busybox ) and or separated in existing module depending on the busybox module being included or not.
That's a good suggestion, I'm just not sure if it worth keep two version of 99squash, the new approach is superior in every way except it depends on busybox. Anyway, I'll do a rebase as there are some conflicts, and try if I can introduce a squash-busybox, and still keep things clean. |
|
Do we need a systemd version check for |
I think that's better be done in the specfile, bump the required systemd version for dracut-squash package, but have to wait until a new systemd release. Without |
|
I've updated the PR, now busybox is a weak dependency, 99squash can work without it, and still keep things simple. |
|
sorry, the latest shellcheck PR rendered this PR unmergeable.. Apologies, that you will have to rebase. |
|
Rebased and check the new change with shellcheck, now it looks good. |
|
@ryncsn you need to fix your commit messages so they pass the commisery check. Good |
Thanks, I'll fix these issues. |
Any other shell should just work.
|
@ryncsn you still need to fix your commit message to pass the commisery check. |
|
Yes, I just notices that. Is there anyway I can do the commisery check locally before pushing it? |
There are instructions here [1] but in most cases people ( including myself ) have done the same mistakes you just did and once you get passed that + avoid including the module number you should be fine on your PR to dracut. The failure messages of the commisery check is pretty self explanatory, if you just read and fix based on the output of the error msg of test itself, you dont need to install/setup anything locally. |
No function change, just move the post install code to 99squash to clean up dracut.sh.
Ensure dracut squash module doesn't effect other steps, and stripping can cover the new binaries installed by it.
Seperate of init.sh and setup-squash.sh is pointless, merge them into one.
Separately install the modules required for squash image setup. These modules can be deleted after squash image setup to save memory. Signed-off-by: Kairui Song <kasong@redhat.com>
The squasn mount points are recorded in text file so later clean up script can umount them, this is not needed, the mount points are fixed so just detect the umount by hardcoded pattern.
Simplify the squash mount layout. Instead of overlay on each top directory (/etc, /usr), just mount and switch_root into the squash image, with a overlay on top of it. Also install the binaries and setup scripts separately, so the squash setup code and the squash image content is independent of each other, all squash setup script and binaries can be deleted safely upon switch_root. With this change, previous squash clean up service and other tricky implementations are all gone. This commit depends on systemd commits from: systemd/systemd#18124 Previouly systemd doesn't recognize non-ramfs initramfs, now this is doable with SYSTEMD_IN_INITRD=lenient Signed-off-by: Kairui Song <kasong@redhat.com>
Use busybox can help reduce the size of early setup environment. With this change, everything is packed in the squash image, and the setup files will be dropped once squash image setup is done, so initramfs stage memory usage is reduced to the minimun, and initramfs decompress is also faster. File layout of a squash initramfs looks like this: ======================================================================== drwxr-xr-x 1 root root 0 Feb 15 14:07 . -rwxr-xr-x 1 root root 946 Feb 15 14:07 init lrwxrwxrwx 1 root root 7 Feb 15 14:07 lib -> usr/lib drwxr-xr-x 1 root root 0 Feb 15 14:07 squash -rw-r--r-- 1 root root 91000832 Feb 15 14:07 squash-root.img drwxr-xr-x 1 root root 0 Feb 15 14:07 usr drwxr-xr-x 1 root root 0 Feb 15 14:07 usr/bin -rwxr-xr-x 1 root root 1293688 Jul 27 2020 usr/bin/busybox lrwxrwxrwx 1 root root 7 Feb 15 14:07 usr/bin/echo -> busybox lrwxrwxrwx 1 root root 7 Feb 15 14:07 usr/bin/mkdir -> busybox lrwxrwxrwx 1 root root 7 Feb 15 14:07 usr/bin/modprobe -> busybox lrwxrwxrwx 1 root root 7 Feb 15 14:07 usr/bin/mount -> busybox lrwxrwxrwx 1 root root 7 Feb 15 14:07 usr/bin/sh -> busybox lrwxrwxrwx 1 root root 7 Feb 15 14:07 usr/bin/switch_root -> busybox drwxr-xr-x 1 root root 0 Feb 15 14:07 usr/lib drwxr-xr-x 1 root root 0 Feb 15 14:07 usr/lib/dracut -rw-r--r-- 1 root root 23 Feb 15 14:07 usr/lib/dracut/build-parameter.txt -rw-r--r-- 1 root root 31 Feb 15 14:07 usr/lib/dracut/dracut-051-93.git20210215.fc33 -rw-r--r-- 1 root root 358 Feb 15 14:07 usr/lib/dracut/modules.txt -rw-r--r-- 1 root root 0 Feb 15 14:07 usr/lib/dracut/need-initqueue drwxr-xr-x 1 root root 0 Feb 15 14:07 usr/lib/modules drwxr-xr-x 1 root root 0 Feb 15 14:07 usr/lib/modules/5.10.11-200.fc33.x86_64 drwxr-xr-x 1 root root 0 Feb 15 14:07 usr/lib/modules/5.10.11-200.fc33.x86_64/kernel <... kernel module misc files skipped ... > ========================================================================
No function change, decide which compressor to use right before compressing the initramfs. This may delay the print of this message: "dracut: no compression tool available. Initramfs image is going to be big." but should be OK, this message is not an error.
Try to make mksquashfs follow --compress option if squash module is included, if not specified or invalid, fall back to use mksquashfs's default compressor.
With squash module, the initramfs is double compressed, which slow down the build progress and doesn't shrink the size much.
Note: this PR depends on systemd/systemd#18124, so I'm not sure when this could be merged, if acceptable.
Current problems
The aim of 99squash module is to reduce the memory usage of initramfs, but there are several problems with the 99squash module:
Changes
With this PR, the 99squash module is changed in following way:
Now the 99squash can be considered an alternative way of compressing the initramfs.
Instead of compress the initramfs image, and decompress everything to ram upon boot, 99squash make the intiamfs content compressed, and decompressed upon file access. This have some advantage on performance and memory usage.
Example initamfs content with the new 99squash (and also a lot of other module enabled for testing):
Note all initramfs content is packed in squash-root.img, and consume 92M of RAM during boot (previously 180M).
This is mostly used for kdump, and the kdump test result is also improved compared to previous implementation.
Checklist
Fixes #