Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zswap: loading modules to late #44901

Open
ikervagyok opened this issue Aug 11, 2018 · 16 comments
Open

zswap: loading modules to late #44901

ikervagyok opened this issue Aug 11, 2018 · 16 comments

Comments

@ikervagyok
Copy link
Contributor

Issue description

At least these missing in the early steps of the boot process:

  • lz4
  • z3fold

This is the same issue as #18533. There is of course the easy solution of compiling those modules into the kernel, but other distributions seem to be able to do this with modules - so what are we doing different in our early startup scripts? Is it just a problem with zswap, or are there other modules (fs, md/dm, ...) affected?

I'll look into this later, but im happy about any leads people give me.

Steps to reproduce

boot.kernelParams = [ "zswap.enabled=1" "zswap.compressor=lz4" "zswap.zpool=z3fold" ]
boot.initrd.kernelModules = [ "lz4" "z3fold" ];
$ dmesg | grep -i z3fold
[    0.252168] zswap: zpool z3fold not available, using default zbud
[    0.822065] stage-1-init: loading module z3fold...
$ dmesg | grep -i lz4
[    0.252120] zswap: compressor lz4 not available, using default lzo
[    0.818979] stage-1-init: loading module lz4...
$ echo lz4 > /sys/module/zswap/parameters/compressor
$ echo z3fold > /sys/module/zswap/parameters/zpool
$ grep -R . /sys/module/zswap/parameters
/sys/module/zswap/parameters/same_filled_pages_enabled:Y
/sys/module/zswap/parameters/enabled:Y
/sys/module/zswap/parameters/max_pool_percent:25
/sys/module/zswap/parameters/compressor:lz4
/sys/module/zswap/parameters/zpool:z3fold

Technical details

nix-shell -I nixpkgs=/nixpkgs -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.17.12, NixOS, 18.09.git.5653b12 (Jellyfish)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.0.4`
 - channels(root): `"nixos-18.03pre130932.cc4677c36ee"`
 - channels(bali): `"nixos-18.09pre140958.696c6bed4e8"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
@dezgeg
Copy link
Contributor

dezgeg commented Aug 12, 2018

Yes, if you set zswap.compressor=lz4 and zswap.zpool=z3fold with kernelParams, and lz4 or z3fold are modules, that's going to always happen. I don't see why it wouldn't be the same case on other distros.

But as you've found out, you can set both of those in /sys/module/zswap/parameters/, so just simply don't use kernelParams to set them.

@ikervagyok
Copy link
Contributor Author

@dezgeg This seems to work on other distros (ubuntu,arch). That's why i believe our boot-up has some ordering issue that will pop up every now and then. i could just override my kernel config, if I was sure that (early) loading of kernel modules works exactly the same as everywhere else. Maybe i'll install arch next to nixos, to test this myself.

@dezgeg
Copy link
Contributor

dezgeg commented Aug 13, 2018

Huh. I guess what can happen then is the kernel can request module loading even before the initrd init script is running. And the probable reason why that doesn't work in NixOS is the initrd doesn't contain a /sbin/modprobe which is what the kernel tries to execute.

@stale
Copy link

stale bot commented Jun 4, 2020

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse.
  3. Ask on the #nixos channel on irc.freenode.net.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 4, 2020
@lunik1
Copy link
Contributor

lunik1 commented Oct 19, 2020

Still important to me. It's easy enough to work around but it would be nice if it wasn't an issue at all.

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Oct 19, 2020
@ikervagyok
Copy link
Contributor Author

ikervagyok commented Oct 23, 2020

@lunik1 the way forward would be a PR to either:

  1. on nixos:
    a. include zbud, z3fold and zsmalloc in the kernel (not as a module)
    b. make a systemd-swap based module
  2. on upstream kernel:
    a. fix the 'bug' that modules that aren't necessary for booting are still loaded as early as possible without a reasonable retry policy (e.g..: after initrd is available)

I've set up a kernel override for all of my machines and since i'm recompiling a few custom packages it doesn't really add a lot of overhead for me. i have no time to investigate this, but 1.b seems the most reasonable solution.

@lunik1
Copy link
Contributor

lunik1 commented Oct 23, 2020

Were you able to figure what specifically about NixOS causes this problem as arch and ubuntu don't have the issue?

@ikervagyok
Copy link
Contributor Author

i think they had these modules built in. setting this up in userspace after loading the os is the sane way of doing it for now. we should join the systemd-swap bandwagon.

@kurnevsky
Copy link
Member

Looks like systemd-swap isn't a good choice anymore: nefelim4ag/systemd-swap@e36ed09

@stale
Copy link

stale bot commented Sep 21, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Sep 21, 2021
@SchrodingerZhu
Copy link

how to workaround this?

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Feb 22, 2023
@kurnevsky
Copy link
Member

@lunik1
Copy link
Contributor

lunik1 commented Feb 22, 2023

Basically the same thing but using preDeviceCommands https://gitlab.com/lunik1/nix-config/-/blob/master/modules/system/zswap.nix

@SchrodingerZhu
Copy link

It works. Thanks for the wonderful examples.

@kurnevsky
Copy link
Member

@lunik1 is there a reason to apply it via preDeviceCommands? :)

@lunik1
Copy link
Contributor

lunik1 commented Feb 23, 2023

Not really. It will activate earlier, but I doubt it will make any practical difference under realistic circumstances.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants