Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zstd module decompression #43

Merged
merged 3 commits into from
Apr 2, 2021
Merged

Conversation

sirlucjan
Copy link
Contributor

kmod 28 supports modules compressed in zstd format so let's add the same functionality to mkinitcpio.

Signed-off-by: Piotr Gorski lucjan.lucjanov@gmail.com

Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
@nycko123
Copy link

nycko123 commented Jan 9, 2021

after several weeks, works with no failures - locally compiled mkinitcpio-zstd

pacman -Qs "kmod|mkinitcpio"

local/kmod 28-1
local/mkinitcpio-busybox 1.32.1-1
local/mkinitcpio-zstd 29-6

/etc/mkinitcpio.conf

MODULES=(i915)
BINARIES=()
FILES=(/etc/vconsole.conf)
HOOKS=(base udev autodetect keyboard modconf block encrypt filesystems fsck)
COMPRESSION="zstd"
COMPRESSION_OPTIONS=('-10' '-T8' '-c' '-z' '-q')

mkinitcpio -p customkernel
log file

@lingruby-pl
Copy link

pacman -Qs "kmod|mkinitcpio"


local/kmod-zstd 27-4
    Linux kernel module management tools and library
local/kmod-zstd-debug 27-4
    Detached debugging symbols for kmod-zstd
local/mkinitcpio-zstd 29-6
    Modular initramfs image creation utility

/etc/mkinitcpio.conf

MODULES=(ext4)
HOOKS=(base)
COMPRESSION="zstd"
COMPRESSION_OPTIONS=('-10' '-T8' '-c' '-z' '-q')

works correctly

functions Outdated
esac
done
(( ${#xz_comp[*]} )) && xz -d "${xz_comp[@]}"
(( ${#gz_comp[*]} )) && gzip -d "${gz_comp[@]}"
(( ${#zst_comp[*]} )) && zstd -d "${zst_comp[@]}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe zstd should be silenced here with the -q option in order to avoid garbage during initramfs regeneration both in console and pacman log.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right, I overlooked that.

Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
@sirlucjan
Copy link
Contributor Author

Thanks for the suggestion, done.

@zhmars
Copy link

zhmars commented Mar 19, 2021

Correct me if I'm wrong. I believe the --rm option is also needed here to remove source module files, like xz/gzip does.

@sirlucjan
Copy link
Contributor Author

sirlucjan commented Mar 19, 2021

I'm pretty sure it's not necessary in this case.

@pfactum What do you think about this?

Adding this will not be the slightest problem.

@zhmars
Copy link

zhmars commented Mar 19, 2021

-k, --keep: keep source file(s) after successful compression or decompression. This is the default behavior.

It seems the source files are preserved by default during initramfs regeneration.
With the --rm option, we can get a smaller initramfs file. (for me, 11M -> 8.5M)

Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
@sirlucjan
Copy link
Contributor Author

@zhmars Done.

@pfactum
Copy link

pfactum commented Mar 19, 2021

@sirlucjan agree with this, it seems that zstd has got inverted defaults unlike gzip and xz.
@zhmars thanks for noticing this.

sudipm-mukherjee pushed a commit to sudipm-mukherjee/linux-test that referenced this pull request Apr 1, 2021
kmod 28 supports modules compressed in zstd format so let's add this
possibility to kernel.

Requires this update to mkinitcpio:
archlinux/mkinitcpio#43

Link: https://lkml.kernel.org/r/20210330113235.2767216-1-lucjan.lucjanov@gmail.com
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
Reviewed-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
@pfactum
Copy link

pfactum commented Apr 1, 2021

@grazzolini please note we are pushing related work to the upstream kernel (https://github.com/hnaz/linux-mm/commit/bb63820634e41278399f3d65ca0f8306eca67351) so that this pull request is expected to be actually usable without kernel patching soon.

@sirlucjan
Copy link
Contributor Author

@pfactum @grazzolini

The patch has been merged into linux-next so perhaps this will appear in 5.13

https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=7b35c1a01c2e2f04638312c4051cf30003dd406d

@grazzolini grazzolini merged commit 51ea831 into archlinux:master Apr 2, 2021
@grazzolini
Copy link
Member

I have just merged this. However, this doesn't mean Arch kernels will start compressing modules with zstd as soon as 5.13 lands.

johnmart19 pushed a commit to ArchiveRepositories/Redline_kernel_ginkgo that referenced this pull request Apr 10, 2021
kmod 28 supports modules compressed in zstd format so let's add this
possibility to kernel.

Requires this update to mkinitcpio:
archlinux/mkinitcpio#43

Link: https://lkml.kernel.org/r/20210330113235.2767216-1-lucjan.lucjanov@gmail.com
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
Reviewed-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
johnmart19 pushed a commit to ArchiveRepositories/Redline_kernel_ginkgo that referenced this pull request Apr 10, 2021
kmod 28 supports modules compressed in zstd format so let's add this
possibility to kernel.

Requires this update to mkinitcpio:
archlinux/mkinitcpio#43

Link: https://lkml.kernel.org/r/20210330113235.2767216-1-lucjan.lucjanov@gmail.com
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
Reviewed-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants