SquashFS picking a compression algorithm

[siboehm]

When I implemented SquashFS-support for conda-pack I only did some rudimentary tests regarding the runtime of packing & unpacking with the different compression options of mksquashfs.

I finally got around to running slightly more detailed performance tests.
I ran the perftests on a project that I had lying around with a conda env of size ~1GB:

1. conda-pack the environment to SquashFS
2. Mount the packed environment in writeable mode using squashfs + overlayfs.
3. source mountpoint/activate && conda-unpack && pip install -e . && pytest && source mountpoint/deactivate.

Compression	Archive size in MB	runtime conda-pack	Runtime activate & install & pytest
no compression	890	0:05	0:57
gzip	263	0:24	0:59
zstd -Xcompression-level 1	286	0:05	0:58
zstd -Xcompression-level 1 -b 262144 -noI -noX	279	0:05	0:58
zstd -Xcompression-level 8	251	0:10	0:57
zstd -Xcompression-level 8 -b 262144 -noI -noX	244	0:12	0:57
zstd -Xcompression-level 16	238	0:45	0:57
zstd -Xcompression-level 16 -b 262144 -noI -noX	230	0:49	0:57
zstd -Xcompression-level 16 -b 262144	228	0:50	0:57
zstd -Xcompression-level 16 -noI -noX	239	0:45	0:57
zstd -Xcompression-level 16 -noI	239	0:43	0:56
zstd -Xcompression-level 16 -noX	238	0:44	0:57
zstd -Xcompression-level 22	236	1:54	0:57
zstd -Xcompression-level 22 -b 262144 -noI -noX	229	1:35	0:57
lz4	387	0:05	0:56
lz4 -b 262144 -noI -noX	385	0:05	0:56
xz	219	0:54	1:07
no squashFS (tar.gz)	280	0:11	1:01
no conda-pack	980	0:00	0:54

• noI: don't compress inode table
• noX: Don't compress extended attrs

For reference, Xar uses zstd -Xcompression-level 16 -b 262144 -noI -noX. libzstd comes pre-installed on Ubuntu and Fedora, it's on brew and conda-forge, too.

Unless the maintainers think otherwise I'll open a PR to use zstd with varying levels of -Xcompression-level based on conda-pack --compress-level, only falling back to xz at the highest level. zstd -Xcompression-level 8 seems like a sane default.