Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem enabling zstd #50

Closed
darkdragon-001 opened this issue Oct 9, 2020 · 5 comments · Fixed by #60
Closed

Problem enabling zstd #50

darkdragon-001 opened this issue Oct 9, 2020 · 5 comments · Fixed by #60

Comments

@darkdragon-001
Copy link

I tried to use zstd compression with zram on Fedora 33 with kernel 5.8.14-300.fc33.x86_64.

# /etc/systemd/zram-generator.conf
[zram0]
compression-algorithm = zstd

Restarting via sudo systemctl restart swap-create@zram0.service gives me the following error:

Okt 09 16:12:03 localhost.localdomain systemd[1]: Starting Create swap on /dev/zram0...
Okt 09 16:12:03 localhost.localdomain kernel: Can't allocate a compression stream
Okt 09 16:12:03 localhost.localdomain kernel: zram: Cannot initialise zstd compressing backend
Okt 09 16:12:03 localhost.localdomain zram-generator[3938]: Error: Failed to configure disk size into /sys/block/zram0/disksize
Okt 09 16:12:03 localhost.localdomain zram-generator[3938]: Caused by:
Okt 09 16:12:03 localhost.localdomain zram-generator[3938]:     Cannot allocate memory (os error 12)
Okt 09 16:12:03 localhost.localdomain systemd[1]: swap-create@zram0.service: Main process exited, code=exited, status=1/FAILURE

I also tried rebooting and the same problem occurs.

$ cat /sys/block/zram0/comp_algorithm
lzo [lzo-rle] lz4 lz4hc 842 zstd

What is going on? Any help appreciated!
@nabijaczleweli
Copy link
Collaborator

nabijaczleweli commented Oct 9, 2020 •
edited

Does this also happen when you run this as root?

modprobe -r zram
modprobe zram
# Give it a moment to settle
echo 1 > /sys/block/zram0/reset
echo zstd > /sys/block/zram0/comp_algorithm
echo $NUMBER > /sys/block/zram0/disksize
/usr/lib/systemd/systemd-makefs swap /dev/zram0

(Substitute $NUMBER for your desired disksize, of course.)

If so, pick another algorithm: does it also happen then?

@darkdragon-001
Copy link
Author

@nabijaczleweli Thanks for your help! modprobe zram solved the problem (which I persisted across reboots via /etc/modules-load.d/). I was just confused because I use btrfs with zstd compression for my root filesystem (from which the systemd config files are loaded) - obviously this doesn't need the additional kernel module...

@polarathene polarathene mentioned this issue Jan 12, 2021
Closed
@keszybz
Copy link
Member

keszybz commented Jan 12, 2021

Since aa03047 (which is included in the zram-generator version in F33, 0.2.0), we do modprobe zram internally. So loading zram should be a noop and shouldn't really change anything. One explanation would be that selinux is preventing the modprobe from being done from the generator. Please check if you had any avcs.

keszybz added a commit to keszybz/zram-generator that referenced this issue Jan 12, 2021
@keszybz
generator: call modprobe for compression modules
2297c66 
Apparently the kernel sometimes doesn't load the appropriate
compression algorithm (see systemd#50, systemd#53). Let's do an explicit modprobe
call, similarly to what we do for the zram module itself.

The operation is minimalistic, and we only load the modules for algs
that were explicitly configured (on the assumption that the default
alg has to be loaded already), and not known to the kernel.

Hopefully fixes systemd#53.
keszybz added a commit to keszybz/zram-generator that referenced this issue Jan 13, 2021
@keszybz
generator: call modprobe for compression modules
cd44ae6 
Apparently the kernel sometimes doesn't load the appropriate
compression algorithm (see systemd#50, systemd#53). Let's do an explicit modprobe
call, similarly to what we do for the zram module itself.

The operation is minimalistic, and we only load the modules for algs
that were explicitly configured (on the assumption that the default
alg has to be loaded already), and not known to the kernel.

Hopefully fixes systemd#53.
@polarathene polarathene mentioned this issue Jan 14, 2021
Merged
keszybz pushed a commit to polarathene/zram-generator that referenced this issue Jan 14, 2021
@polarathene @keszybz
fix: manually load kernel modules for compression
8903e89 
Creation of a zram device (with zstd as the compression alg) occasionally
fails with ENOMEM:
Jan 13 09:59:39 vultr.guest systemd[1]: Starting Create swap on /dev/zram3...
Jan 13 09:59:39 vultr.guest kernel: Can't allocate a compression stream
Jan 13 09:59:39 vultr.guest kernel: zram: Cannot initialise zstd compressing backend
Jan 13 09:59:39 vultr.guest zram-generator[997]: Error: Failed to configure disk size into /sys/block/zram3/disksize
Jan 13 09:59:39 vultr.guest zram-generator[997]: Caused by:
Jan 13 09:59:39 vultr.guest zram-generator[997]:     Cannot allocate memory (os error 12)

https://bugzilla.kernel.org/show_bug.cgi?id=211173

The kernel normally loads the compressor module when a device with a
given compression module is loaded, but this doesn't always work.
Apparently, loading the compression module up front avoids the issue.

Fixes systemd#50, systemd#53.

Note: /sys/block/zram0/comp_algorithm lists compressors known to zram,
not the ones currently loaded. /proc/crypto lists currently loaded
crypto & compression modules.
keszybz pushed a commit to polarathene/zram-generator that referenced this issue Jan 14, 2021
@polarathene @keszybz
fix: manually load kernel modules for compression
2b5bbdd 
Creation of a zram device (with zstd as the compression alg) occasionally
fails with ENOMEM:
Jan 13 09:59:39 vultr.guest systemd[1]: Starting Create swap on /dev/zram3...
Jan 13 09:59:39 vultr.guest kernel: Can't allocate a compression stream
Jan 13 09:59:39 vultr.guest kernel: zram: Cannot initialise zstd compressing backend
Jan 13 09:59:39 vultr.guest zram-generator[997]: Error: Failed to configure disk size into /sys/block/zram3/disksize
Jan 13 09:59:39 vultr.guest zram-generator[997]: Caused by:
Jan 13 09:59:39 vultr.guest zram-generator[997]:     Cannot allocate memory (os error 12)

https://bugzilla.kernel.org/show_bug.cgi?id=211173

The kernel normally loads the compressor module when a device with a
given compression module is loaded, but this doesn't always work.
Apparently, loading the compression module up front avoids the issue.

Fixes systemd#50, systemd#53.

Note: /sys/block/zram0/comp_algorithm lists compressors known to zram,
not the ones currently loaded. /proc/crypto lists currently loaded
crypto & compression modules.
@darkdragon-001
Copy link
Author

When I remove the manual module loading, it seems to be still working. Thanks for your help!

@keszybz
Copy link
Member

keszybz commented Feb 2, 2021

@darkdragon-001 if you could provide the debug information that the kernel devs requested in https://bugzilla.kernel.org/show_bug.cgi?id=211173, that'd be super cool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Load kernel module for compressors when required polarathene/zram-generator
3 participants
@darkdragon-001 @keszybz @nabijaczleweli