Fix broken TIFF reads of multi-subimage non-spectral files (AcademySo…

…ftwareFoundation#2692)

For a few rare edge cases of photometric interpretation, we fall back
on libtiff's ability to auto-translate to an RGBA image for us. It
only does this for the whole image at once, so we do it for the first
scanline we read (storing the whole image in a buffer, skipping the
conversion for subsequent scanlines -- and it can tell it's the
"first" scanline because the buffer isn't yet allocated).

The bug was that for these cases (such as a photometric YCbCr), if the
file was *multi-subimage*, we were not clearing the rgba buffer, so
upon reading the second subimage, it thought its buffer was already
filled, so it wasn't actually doing the read and convert, just using
whatever was already living in the buffer. Oops.

Also, I noticed that we should clear m_use_rgba_interface for each
subimage -- it was previously just setting it when it found those edge
cases. We never saw a symptom from this, but I realized it would be
problematic for a file where the first subimage was YCbCr and the
second subimage was regular RGB, it wouldn't know to fall back to our
usual code path.

Signed-off-by: Larry Gritz <lg@larrygritz.com>

src/tiff.imageio/tiffinput.cpp

@@ -176,8 +176,8 @@ class TIFFInput final : public ImageInput {
         if (m_tif) {
             TIFFClose(m_tif);
             m_tif = NULL;
-            if (m_rgbadata.size())
-                std::vector<uint32_t>().swap(m_rgbadata);  // release
+            m_rgbadata.clear();
+            m_rgbadata.shrink_to_fit();
         }
     }
 
@@ -680,14 +680,16 @@ TIFFInput::seek_subimage(int subimage, int miplevel)
         readspec(read_meta);
         // OK, some edge cases we just don't handle. For those, fall back on
         // the TIFFRGBA interface.
-        bool is_jpeg        = (m_compression == COMPRESSION_JPEG
+        bool is_jpeg         = (m_compression == COMPRESSION_JPEG
                         || m_compression == COMPRESSION_OJPEG);
-        bool is_nonspectral = (m_photometric == PHOTOMETRIC_YCBCR
+        bool is_nonspectral  = (m_photometric == PHOTOMETRIC_YCBCR
                                || m_photometric == PHOTOMETRIC_CIELAB
                                || m_photometric == PHOTOMETRIC_ICCLAB
                                || m_photometric == PHOTOMETRIC_ITULAB
                                || m_photometric == PHOTOMETRIC_LOGL
                                || m_photometric == PHOTOMETRIC_LOGLUV);
+        m_use_rgba_interface = false;
+        m_rgbadata.clear();
         if ((is_jpeg && m_spec.nchannels != 3)
             || (is_nonspectral && !m_raw_color)) {
             char emsg[1024];

testsuite/tiff-suite/ref/out-alt.txt

@@ -88,7 +88,7 @@ Reading ../../../../../libtiffpic/dscf0013.tif
     tiff:PlanarConfiguration: 1
     tiff:RowsPerStrip: 15
  subimage  1:  160 x  120, 3 channel, uint8 tiff
-    SHA-1: D7C5E398DC47C1FF619F62650EE2F8B303ADCDDC
+    SHA-1: A273D70F123360DE64EB07748E7EED98486C7F4B
     channel list: R, G, B
     compression: "none"
     Orientation: 1 (normal)

testsuite/tiff-suite/ref/out-alt2.txt

@@ -88,7 +88,7 @@ Reading ../../../../../libtiffpic/dscf0013.tif
     tiff:PlanarConfiguration: 1
     tiff:RowsPerStrip: 15
  subimage  1:  160 x  120, 3 channel, uint8 tiff
-    SHA-1: D7C5E398DC47C1FF619F62650EE2F8B303ADCDDC
+    SHA-1: A273D70F123360DE64EB07748E7EED98486C7F4B
     channel list: R, G, B
     compression: "none"
     Orientation: 1 (normal)

testsuite/tiff-suite/ref/out-jpeg9b.txt

@@ -88,7 +88,7 @@ Reading ../../../../../libtiffpic/dscf0013.tif
     tiff:PlanarConfiguration: 1
     tiff:RowsPerStrip: 15
  subimage  1:  160 x  120, 3 channel, uint8 tiff
-    SHA-1: D7C5E398DC47C1FF619F62650EE2F8B303ADCDDC
+    SHA-1: A273D70F123360DE64EB07748E7EED98486C7F4B
     channel list: R, G, B
     compression: "none"
     Orientation: 1 (normal)

testsuite/tiff-suite/ref/out.txt

@@ -88,7 +88,7 @@ Reading ../../../../../libtiffpic/dscf0013.tif
     tiff:PlanarConfiguration: 1
     tiff:RowsPerStrip: 15
  subimage  1:  160 x  120, 3 channel, uint8 tiff
-    SHA-1: D7C5E398DC47C1FF619F62650EE2F8B303ADCDDC
+    SHA-1: A273D70F123360DE64EB07748E7EED98486C7F4B
     channel list: R, G, B
     compression: "none"
     Orientation: 1 (normal)