Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
scripts/image: use maximum lzo and zstd compression level for images #2892
The compression methods used by mksquashfs take compression levels, which are trade offs between time spent compression and size of the compressed file. This makes use of them for lzo and zstd. For lzo, it's a ~40kb size savings, and for zstd it's about 500kb size savings.
I also tested gzip, where the size reduction was 4kb, and xz where size reduction was 8kb. If those reductions should be included, let me know. gzip is a simple -o squashfs_compression == gzip to the lzo test, while xz requires exporting a variable.
Could you add the gzip support, as we use that for Generic. Could you add it as:
as that will ensure the compression level is applied even when the compression method isn't configured.
(Note that we don't use
We don't currently use xz with any projects (and I don't think we have any plans to do so at this time) so there'd be no point adding extra complexity for something we don't use. That said, we probably will use zstd in future, so that's a nice to have.
@antonlacon Remember higher compression levels are mostly resulting in higher cpu/time usage at extraction. Generic is powerful enough for everything, RPix and even S905 have problems with the basic compression levels. Difficult to predict if the additional time matters or not. Pls do some time comparison too (how long the RPi... takes to read the squashfs).
I used a script to copy test data to a ramdisk, and then create multiple squashfs images with different settings. Those images were then mounted and had the files within cat'ed to /dev/null. The test data was a filesystem built from an LE-9 git pull from earlier this week without debug. Symlinks and empty directories were deleted from /, leaving /etc, /opt and /usr. Its uncompressed size was about 303MB.
Software libraries to build the images:
Results from an RPi3 running LE-9 git from earlier this week built for aarch32 off an sd card. It was running kodi (idle) and whatever else LE runs in the background active during the test. Compression level 8 is the default for gzip and lzo when invoked by mksquashfs. 15 is the default for zstd.
Uncompressed was done by hand (same command as script). The rest were by scripts. The scripts are available here: https://github.com/antonlacon/squashfs-experiments/tree/LE-testing
Edit the scripts for desired block size and compression level to test, or run on all 240 combinations. Images are built with test-mksquashfs.sh, and tested on the LE device with test-mksquashfs-device.sh. report.sh outputs the results to csv.
I can't recall any concerns with
I'm OK with this change and have had it in my test builds for a week or so, and I think we should merge it.