Can't open a tiff in 3.6.0dev that opens in 3.4.3

[Houska1]

Expected behavior and actual behavior.

I've created and used a tiff file (in GDAL and QGIS earlier versions) that opens without trouble in GDAL through 3.4.3, but won't open in 3.6.0dev.

File temporarily available online at https://www.dropbox.com/s/jnu5iwg5ehlll34/DRAPE%202014%20IR%20compressed.tif?dl=0 while this ticket is open. [50 Meg]

Steps to reproduce the problem.

gdal_translate --debug on "DRAPE 2014 IR compressed.tif" "test.tif"
GDAL: GDALOpen(DRAPE 2014 IR compressed.tif, this=0000024646166730) succeeds as GTiff.
GDAL: Using GTiff driver
Input file size is 16788, 13621
GDAL: Using GTiff driver
GTiff: ScanDirectories()
GTiff: Opened 8394x6811 overview.
GTiff: Opened 4197x3406 overview.
GTiff: Opened 2099x1703 overview.
GTiff: Opened 1050x852 overview.
0GDAL: GDAL_CACHEMAX = 806 MB
GDAL: GDALDatasetCopyWholeRaster(): 16788*198 swaths, bInterleave=1
ERROR 1: JPEGLib:Unsupported JPEG data precision 8
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: DRAPE 2014 IR compressed.tif, band 1: IReadBlock failed at X offset 0, Y offset 0: TIFFReadEncodedTile() failed.
GDAL: GDALClose(DRAPE 2014 IR compressed.tif, this=0000024646166730)
GDAL: In GDALDestroy - unloading GDAL shared library.

The IReadBlock errors also appear in loading this file into QGIS, via an existing project or new project (which is what matters to me, above is just a minimal error case to replicate).

Ditto happens for some other rasters created similarly; not unique to this file.

Operating system

Windows 10

GDAL version and provenance

GDAL 3.6.0dev-efd4398104, OSGeo4W. Has been present since intro of 3.6.0dev into OSGeo4W QGIS dev dailies. Not sure about 3.5.0. Released 3.4.3 and previous work fine.

I initially throught this might be some compilation flag/config error, so reported at https://trac.osgeo.org/osgeo4w/ticket/740 a week ago, but no answer/solution there. Thanks!

[rouault]

@jef-n I suspect a build issue in OSGeo4W where libjpeg12 would be used in (internal?) libtiff
Maybe JPEG_INCLUDE_DIR catches the libjpeg.h from libjpeg12 ?

[jef-n]

See also #5489

[jratike80]

I had a try with a GDAL 3.6.0dev, released 2022/99/99 build by gisinternals https://download.gisinternals.com/sdk/downloads/release-1930-x64-gdal-mapserver.zip from 2022-05-06 (there seems to be build errors since that). Works fine

GDAL: GDALOpen(DRAPE 2014 IR compressed.tif, this=00000237AE1BA980) succeeds as GTiff.
GDAL: Using GTiff driver
Input file size is 16788, 13621
GDAL: Using GTiff driver
GTiff: ScanDirectories()
GTiff: Opened 8394x6811 overview.
GTiff: Opened 4197x3406 overview.
GTiff: Opened 2099x1703 overview.
GTiff: Opened 1050x852 overview.
0GDAL: GDAL_CACHEMAX = 1632 MB
GDAL: GDALDatasetCopyWholeRaster(): 16788*198 swaths, bInterleave=1
...10...20...30...40...50...60...70...80...90...100 - done.
GDAL: Flushing dirty blocks: 0...10...20...30...40...50...60...70...80...90...100 - done.
GDAL: GDALClose(test.tif, this=00000237AE1B97B0)
GDAL: GDALClose(DRAPE 2014 IR compressed.tif, this=00000237AE1BA980)
GDAL: In GDALDestroy - unloading GDAL shared library.

[rouault]

See also #5489

completely unrelated. The error reported in this ticket seems to be a configuration issue of the build related to a mix of libjpeg 8 bit vs libjpeg 12 bit

[jef-n]

@jef-n I suspect a build issue in OSGeo4W where libjpeg12 would be used in (internal?) libtiff Maybe JPEG_INCLUDE_DIR catches the libjpeg.h from libjpeg12 ?

Building with https://github.com/jef-n/OSGeo4W/blob/cfce156556836e1e18b370fe1bfb7f845bb3703b/src/gdal-dev/osgeo4w/package.sh#L173 - so it should be using external libs. gdalinfo --format GTiff from 3.4 and 3.6 only has these differences:

-    LIBGEOTIFF=1700
-    LIBTIFF=INTERNAL
+    LIBGEOTIFF=1710
+    LIBTIFF=LIBTIFF, Version 4.3.0
+Copyright (c) 1988-1996 Sam Leffler
+Copyright (c) 1991-1996 Silicon Graphics, Inc.

[rouault]

https://github.com/jef-n/OSGeo4W/blob/cfce156556836e1e18b370fe1bfb7f845bb3703b/src/gdal-dev/osgeo4w/package.sh#L173 so it should be using external libs

line 173 only disables the use of libjpeg internal, but internal libtiff can potentially use either internal or external libjpeg. That said, the diff you show indicates the use of external libtiff, so maybe the misconfiguration is in the external libtiff ?

[jef-n]

Not sure how that all fits together. This links. I ran into issues linking poppler, that uses libtiff too, earlier (and had to introduce GDAL_EXTRA_LINK_LIBRARIES on the way)

[rouault]

ok looking at the build recipee of libtiff in https://github.com/jef-n/OSGeo4W/blob/master/src/libtiff/osgeo4w/package.sh , I see it is configured with libjpeg and libjpeg12.
Looking at https://github.com/jef-n/OSGeo4W/blob/master/src/libjpeg/osgeo4w/package.sh , I see some effort to avoid clashing symbols between libjpeg and libjpeg12, but I'm not sure how robust that is.
I would suspect that libtiff is confused and doesn't include the appropriate jpeglib.h
If I remember correctly an investigation of a few months ago, my conclusion was it was impractical to build libtiff against libjpeg12 .
My recommendation would be that:

• you enable libtiff and libgeotiff in GDAL (GDAL does the appropriate symbol renaming to avoid clashes). Internal libtiff can use GDAL internal libjpeg12 (which does symbol renaming to avoid clashes with internal/external libjpeg)
• and/or you drop libjpeg12 support in external libtiff
  As an enhancement, I would strongly suggest using libjpeg-turbo for the external libjpeg, rather than libjpeg-9.

[jef-n]

thanks. switched to jpeg-turbo and rebuilt (and partly updated) reverse dependencies in OSGeo4W.