Add libdeflate support for accelerated (35-50%) internal libtiff ZIP/Deflate compression/decompression

[rouault]

The first commit improves the ZIP/Deflate codec of internal libtiff to be able to use libdeflate (https://github.com/ebiggers/libdeflate). It has been submitted to upstream libtiff (per https://gitlab.com/libtiff/libtiff/-/merge_requests/158), so ultimately the improvement will be available when using external libtiff as well.
It is accompanied by the needed changes in the GTiff driver, autotest and in the GDAL build system (autoconf/nmake) to be able to build against libdeflate.

So we can have 2 kind of builds with the Zip/Deflate codec:

• zlib only
• zlib + libdeflate

Speed improvements in the 35%-50% range can be expected when libdeflate is used, that is when reading/writing a full tile or stip (which is the nominal situation for well behaved TIFF files). Compression level up to 12 is now supported (capped to 9 when zlib is used).
Still requires zlib for situations where libdeflate cannot be used (that is for scanline access, since libdeflate has no streaming mode)
Of course, deflate codestreams produced by libdeflate can be read by zlib, and vice-versa.

Funded by EOX and Safe software

---

The second commit adds usage of libdeflate to GDAL itself in the CPLZlibDeflate() and CPLZLibInflate() functions. So having an explicit GDAL libdeflate-enabled build is required to benefit from that, independently of the choice of internal vs external libtiff
Similarly to the libtiff case, zlib is still needed, since libdeflate cannot be always used. This could improve a bit the speed of the MVT and OSM PBF drivers (but probably not that much, at least for the OSM PBF case)

[rouault]

Interesting fact. Deflate with libdeflate can be, at least in some circumstances, faster than ZSTD.

Source file: world.topo.bathy.200406.3x21600x21600.B2.png (21600 x 21600 pixels, 3 bands, 8bits)
Converted into a tiled (256x256) deflate compressed TIFF (deflate.tif) and a tiled (256x256) ZSTD compressed TIFF (zstd.tif)

Single threaded deflate->deflate ans zstd->zstd:

• gdal_translate deflate.tif deflate_out.tif -co tiled=yes -co compress=deflate: 31 s (with libdeflate 1.5)
• gdal_translate zstd.tif zstd_out.tif -co tiled=yes -co compress=zstd: 37 s (with zstd 1.4.5. it is slightly faster with zstd 1.0.0)
• gdal_translate deflate.tif deflate_out.tif -co tiled=yes -co compress=deflate --config GDAL_TIFF_DEFLATE_SUBCODEC ZLIB: 58 s (with zlib)

8 threads (for compression):

• gdal_translate deflate.tif deflate_out.tif -co tiled=yes -co compress=deflate -co num_threads=8: 10 s (total all cpus: 46s)
• gdal_translate zstd.tif zstd_out.tif -co tiled=yes -co compress=zstd -co num_threads=8: 14 s (total all cpus: 1m17s)
• gdal_translate deflate.tif deflate_out.tif -co tiled=yes -co compress=deflate -co num_threads=8 --config GDAL_TIFF_DEFLATE_SUBCODEC ZLIB: 17s (total all cpus:1m28s)

And the deflate file (with libdeflate) is slightly smaller than the zstd one (but of course that's just with the default level): 265 MB vs 275 MB.
That maybe an artifact of that file being super compressible (~ 5 compression factor)

Using -co PREDICTOR=2 doesn't change fundamentally things: it adds extra processing time in both cases and reduce file size to 232 MB (libdeflate) and 234 MB (zstd)

Actually testing only the decompression side with python -c "from osgeo import gdal; ds = gdal.Open('source.tif'); ds.ReadAsArray()" shows that the decompression spead between zstd and libdeflate is very close (~ 4s)

It might be that the ZSTD codec in libtiff is not fully optimized, or just a specificity of that kind of file / compression ratio.