mozjpeg doesn't honour non-optimized coding mode of libtiff

[rouault]

It has been found that in the default mode of libtiff for JPEG compression, that uses default Huffman tables stored in the JpegTables TIFF tag (more details about that tag can be found in the "JPEGTables field" paragraph of https://www.awaresystems.be/imaging/tiff/specification/TIFFTechNote2.txt), and use non-optimized coding for the TIFF tile/strips, mozjpeg still emits Huffman tables in the TIFF tile/strips, which breaks the decoding

This can be reproduced when building libtiff master (https://gitlab.com/libtiff/libtiff) against mozjpeg, after having removed the if () block at https://gitlab.com/libtiff/libtiff/-/blob/master/libtiff/tif_jpeg.c#L1908 (or just removing the line that does sp->jpegtablesmode &= ~JPEGTABLESMODE_HUFF;), and running tiffcp -c jpeg -t in.tif out.tif on the attached file. The bottom of the resulting file will be corrupted when decoding.
in.tif.zip
corrupted_out.tif.zip

libtiff works fine with libjpeg-turbo, so this is a mozjpeg specific behaviour. A workaround has been implemented in https://gitlab.com/libtiff/libtiff/-/merge_requests/254 to make libtiff use optimized coding systemetically, but this isn't fully satisfactory as the user has normally the control on if he wants optimized coding or default Huffman tables.

[kornelski]

That's a very unfortunate conflict of requirements. I know MozJPEG is supposed to be a drop-in replacement for libjpeg, but OTOH disabling such an essential optimization ruins compression, and defeats the whole point of using MozJPEG.

Is the absence of custom Huffman tables a requirement of the TIFF standard, or is it just a feature in libtiff?

[rouault]

Is the absence of custom Huffman tables a requirement of the TIFF standard, or is it just a feature in libtiff?

The TIFF standard allow both custom Huffman tables or standard. The default of libtiff is to use standard, but the user can decide otherwise (the GDAL library that leverages libtiff for example has changed the default behaviour to use custom Huffman tables, and thus isn't affected by this issue in its default mode)

(Another issue with mozjpeg was that it defaults to use progressive scans, which is disallowed by the TIFF standard, but I could change that easily be setting num_scans = 0 and scan_info = NULL, so there is no longer any issue regarding that point)