scripts/image: use 256kb blocksize for squashfs images

[antonlacon]

Squashfs allows configuring a blocksize between 4KB and 1MB. The default is
128KB. Increasing blocksize, in general, increases compression efficiency
at a cost of increased access time. Using 256KB for a blocksize appears to be
a sweet spot balancing the two.

Gzip decreases image by ~700KB.
Lzo decreases image by ~850KB.
Zstd decreaes image by ~2.5MB.

Benchmarking device is an RPi3 running an AArch32 build of LE-9 git from 9/19/18. Test data is a LE9 git filesystem (the /etc and /usr directories - 303MB) from the end of August; the same used for compression level testing. Filename is squashfs-compression method-compression level-blocksize. UTIME is access time to decompress and dump the image to /dev/null.

Filename	Size	Ratio	UTIME
squashfs-gzip-9-4096.squashfs	141212	49.27%	0m11.37s
squashfs-gzip-9-8192.squashfs	135588	47.33%	0m9.80s
squashfs-gzip-9-16384.squashfs	131064	45.76%	0m9.19s
squashfs-gzip-9-32768.squashfs	126864	44.30%	0m8.58s
squashfs-gzip-9-65536.squashfs	124016	43.31%	0m9.03s
squashfs-gzip-9-131072.squashfs	122516	42.79%	0m8.67s
squashfs-gzip-9-262144.squashfs	121788	42.54%	0m8.81s
squashfs-gzip-9-524288.squashfs	121476	42.43%	0m10.12s
squashfs-gzip-9-1048576.squashfs	121328	42.38%	0m11.34s
squashfs-lzo-9-4096.squashfs	159424	55.62%	0m7.85s
squashfs-lzo-9-8192.squashfs	152772	53.33%	0m7.93s
squashfs-lzo-9-16384.squashfs	146800	51.26%	0m7.71s
squashfs-lzo-9-32768.squashfs	140892	49.20%	0m7.57s
squashfs-lzo-9-65536.squashfs	136336	47.61%	0m7.36s
squashfs-lzo-9-131072.squashfs	133668	46.69%	0m7.42s
squashfs-lzo-9-262144.squashfs	132436	46.26%	0m7.51s
squashfs-lzo-9-524288.squashfs	131860	46.06%	0m7.49s
squashfs-lzo-9-1048576.squashfs	131552	45.95%	0m8.20s
squashfs-zstd-22-4096.squashfs	136072	47.48%	0m11.66s
squashfs-zstd-22-8192.squashfs	129508	45.21%	0m10.38s
squashfs-zstd-22-16384.squashfs	124008	43.30%	0m9.66s
squashfs-zstd-22-32768.squashfs	118996	41.56%	0m9.31s
squashfs-zstd-22-65536.squashfs	115388	40.30%	0m9.18s
squashfs-zstd-22-131072.squashfs	112624	39.34%	0m9.22s
squashfs-zstd-22-262144.squashfs	109632	38.29%	0m9.33s
squashfs-zstd-22-524288.squashfs	106472	37.19%	0m10.20s
squashfs-zstd-22-1048576.squashfs	103540	36.17%	0m10.71s

[antonlacon]

Benchmarking scripts: https://github.com/antonlacon/squashfs-experiments/tree/LE-testing

[MilhouseVH]

Yes, the 256KB blocksize looks pretty good - thanks!

zstd also looks promising, though not entirely sure when we'll switch to that (but I'm sure we will eventually).

[MilhouseVH]

One thought might be to make the blocksize configurable - 256KB may not be the sweetspot for all compression algos. Adding -b 262144 to SQUASHFS_COMPRESSION_OPTION would give more flexibility:

if [ "$SQUASHFS_COMPRESSION"  = "lzo" -o "${SQUASHFS_COMPRESSION:-gzip}" = "gzip" ]; then
  SQUASHFS_COMPRESSION_OPTION="-Xcompression-level 9 -b 262144"
elif [ "$SQUASHFS_COMPRESSION" = "zstd" ]; then
  SQUASHFS_COMPRESSION_OPTION="-Xcompression-level 22 -b 262144"
fi

I know it's the same blocksize for lzo/gzip/zstd but anyone using a different compression algo (eg. xz, lz4 etc.) can then specify their own blocksize and compression level.

In fact we probably only want to set SQUASHFS_COMPRESSION_OPTION if it's not already set, ie.:

if [ -z "$SQUASHFS_COMPRESSION_OPTION" ]; then
  if [ "$SQUASHFS_COMPRESSION"  = "lzo" -o "${SQUASHFS_COMPRESSION:-gzip}" = "gzip" ]; then
    SQUASHFS_COMPRESSION_OPTION="-Xcompression-level 9 -b 262144"
  elif [ "$SQUASHFS_COMPRESSION" = "zstd" ]; then
    SQUASHFS_COMPRESSION_OPTION="-Xcompression-level 22 -b 262144"
  fi
fi

as this would allow Xcompression-level and blocksize to be custom configured even for lzo/gzip/zstd.

And then the change to the mksquashfs line can be dropped.

[antonlacon]

Adjusted and split lzo out to its own test using 512kb blocksize based on the above testing. New image size reduction there is ~1.25MB.

[MilhouseVH]

No issues reported from almost 2 weeks testing RPi/RPi2/Generic, where I saw similar reductions in size as you found. Best way to test this now is to merge it - many thanks!