Add zswap support to Raspberry Pi Kernels #3432
Conversation
…rnels. This adds several kb of kernel size and module size per kernel, but allows the use of zswap (compressed swap fronting) without having to rebuild the kernels. This is quite useful for desktop use of Pi boards.
|
Is there anything required from me to get this merged, or are config changes like this merged at a single point for future kernel releases? |
|
Is there anything else I need to do to have this merged, or is there no interest in it at this point? |
|
I noticed the Raspberry Pi Foundation Engineer's arguments about NOT ENABLING zswap stating that they "didn't find a compelling reason to use zram or zswap", for some tests show improvements and others show degradation in performance on any particular combination of use case & media of their choice. What I don't understand is that their conclusion must also mean "It is better that no user even try to use zswap" -- implied by the fact that the modules are marked as "do not even compile" -- despite the cost of the modules is just ~52k of storage? The alternative -- forcing very determined users to have to build the modules on every kernel update -- seems to be unfair for further research as it restricts (or even impedes) the possibility of counter arguments as this "do not even include it" decision imposes a limit on the number of users who would care enough to test and report use cases where zswap are a better choice than no zswap. Anyway... using this rationale to convince just myself, I'll try to build it as a module and report the improvements here. |
|
I saw your PR adds swap built into the kernel and the other bits as modules. Maybe it would be better to add all of them as modules? So that the only cost of this would be a tiny storage space... |
|
Well, nobody from the Pi foundation has offered an opinion on it, so I'm pretty sure they don't care. |
|
Here I share my results, which favor zswap, in speed, by 21%: The test: on as Raspberry Pi 1 512MiB of RAM, two instances of the following command have been issued at the same time: Using the standard swap (zswap disabled), these commands finished -- each -- with: real 11m31.509s With zswap enabled with the following parameters: I got: real 9m36.542s In any order I run the tests, the results are consistent -- after stopping/starting the swap devices and dropping the caches. As a side note, I've also ran the tests with 5% max pool. It was just marginally better than no zswap: real 11m25.577s About the disk swap usage: My Raspberry Pi 1, 512MiB of RAM, was idle and had 447168k available for the tests (the rest is for the system and gpu 51MiB GPU RAM). Conclusions:
|
|
@popcornmix @pelwell The use of ZSwap has been bubbling around for while now. If its a module with no impact unless enabled (if that is indeed the case), any reason not to have it? |
|
ZSWAP is a boolean not a module, so is likely to affect all users, not just those who enable it. |
|
Before: After PR So kernel.img up by ~10K, modules up by 36K, free mem down by 1344K without enabling zswap. The loss of free memory for everyone is concerning. |
|
I'll try and find out where the RAM is going. |
|
As noted, you can't build zswap as a module, only the various options it has. I didn't think to check the RAM configs, though can try to replicate that. My solution to a minor increase in memory use would be to enable zswap with the stock 100MB backing file, as it'll easily cover the delta as soon as there's any swap pressure, but I understand there's hesitation to do that based on the Pi1 behavior. In my experience, zswap is pretty much a pure win for small values of "maximum memory" - I'd be fine with 10% as the default limit, though I'm not sure how to set that outside config files or the kernel command line. I've found it to be worth doing on any Pi - the same-filled-page compression alone saves quite a bit of memory as swap pressure goes up. How repeatable are the "free" results on boot? I've not actually tested that, but can certainly spend some time on it. |
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some __init functions have to remain in memory after the init phase, just in case they are needed later, but the total size is negligible. See: raspberrypi#3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
|
There's a PR at #3600 that is an attempt at reducing the cost of enabling the ZSWAP config option but not enabling it at runtime. If it gets some testing, and the overhead for the disabled case isn't too bad, and if we can stand the thought of maintaining yet another downstream patch, then it might get merged. |
|
Please don't confuse zram and zswap - they're two entirely different features, entirely different code paths, etc. |
|
How are you getting the free values? Just running it repeatedly on a freshly booted Pi with buster-lite, no network, at the console, I'm seeing over a megabyte difference in values over time. I'll write a script to poll it and average it, but just random sample in time values doesn't seem like the right answer here. |
|
I tried to make my measurements as soon after boot as possible, taking a sequence of 5 or 6 readings and picking the middle value (or the one that looked the most representative) - formal average sounds fine. A figure that excluding filesystem caching would be more stable and representative, but that might need some maths on the contents of /proc/meminfo. |
|
Been testing this patch on my raspberry pi and it runs just fine same as on my cubieboard2, better then regular swapping |
|
Did you test this patch (which won't get merged) or #3600 (which might if it gets enough positive feedback)? |
|
tested this one but i can give the other one a whirl also isnt a big deal to compile |
|
Thanks - it would be appreciated. |
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: raspberrypi#3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: raspberrypi#3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
|
[ I don't understand why Linux's free memory after boot is so variable, even with no network. It makes serious A/B testing of memory usage almost impossible. ] See #3626 for a new version of the deferred initialisation patches. The initialisation has been split so that some can still be done during the init phase, but pool creation can still be deferred. For some reason, building in zbud makes virtually no difference to the kernel size, as measured by the symbol |
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
|
#3626 was merged on the command line, and the latest rpi-update kernel includes it. |
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
|
Well, I'll close this one out - thanks! |
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: raspberrypi#3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: #3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Enabling zswap support in the kernel configuration costs about 1.5MB of RAM, even when zswap is not enabled at runtime. This cost can be reduced significantly by deferring initialisation (including pool creation) until the "enabled" parameter is set to true. There is a small cost to this in that some initialisation code has to remain in memory after the init phase, just in case they are needed later, but the total size increase is negligible. See: raspberrypi/linux#3432 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Context: #2649
Short context: zswap is a compressed/in-memory front end to disk based swap. When a page is to be swapped out, if zswap is enabled, it tries to compress the page (using a configurable compression algorithm) and store it in memory. If the page does not compress well, it is sent to disk, and if the configurable-sized in-memory cache fills up, pages are evicted with a LRU algorithm to disk. On modern kernels with same filled page compression, compression ratios of 2-3x are seen in practice in typical desktop use.
This is distinct from zram, in that it properly evicts old pages to disk - zswap and zram are different features, with different behaviors. The inversion present in zram (where old pages are stored perpetually in memory and fresh pages are evicted to disk) is not an issue in zswap.
Experimentally, on the Raspberry Pi hardware, zswap adds a substantial amount of "desktop usability" when given a properly sized swap file. The Pi3 series goes from "unable to handle more than trivial use" to "usable as a light desktop," and the Pi4 benefits as well, depending somewhat on the amount of RAM present.
This patch does /not/ enable zswap by default, but simply adds the features to the kernel. It adds zswap, zbud (a "two compressed page per page of RAM" storage algorithm), and z3fold (a "three compressed page per page of RAM" storage algorithm). It is up to the user to properly configure the system to use these features, and other than the disk space used (examined below), this does not, in any way, change the system behavior.
====================
Cross compiling on an 64-bit x86 host with the reference Raspberry Pi cross compiler tools, size changes in the kernels/modules across the platforms:
bcmrpi_defconfig:
zImage: 4916kb to 4924kb, +8kb
Image: 10792kb to 10808kb, +16kb
Modules directory: 58632kb to 58644kb, +12kb
bcm2709_defconfig:
zImage: 5188kb to 5192kb, +4kb
Image: 11360kb to 11376kb, +16kb
Modules directory: 58596kb to 58608kb, +12kb
bcm2711_defconfig:
zImage: 5508kb to 5516kb: +8kb
Image: 12064kb to 12080kb: +16kb
Modules directory: 58796kb to 58804kb, +8kb
====================
For a full Raspbian install, this patch adds 52kb storage to the SD card image (zImage + modules), and offers users additional flexibility in how they can configure the system for various uses.
The intent is that these changes propagate forward and are the new defaults for Raspberry Pi kernels.