Cherry-pick PR's into the RB-2.5 branch for v2.5.5 release (#917)

* Revert "Disable OPENEXR_IMF_HAVE_GCC_INLINE_ASM_AVX when building on arm64 macOS" This reverts commit 67053eb. Signed-off-by: Harry Mallon <hjmallon@gmail.com> * Fix Apple Universal 2 (arm64/x86_64) builds * In these types of builds we want arm64 and x86_64 (with AVX optimisations). However the way cmake works (with `CMAKE_OSX_ARCHITECTURES="arm64;x86_64"` means that we share one OpenEXRConfigInternal.h between both builds. So we have to have OPENEXR_IMF_HAVE_GCC_INLINE_ASM_AVX mean "AVX GCC asm is available if platform is x86", rather than "AVX GCC asm is available". Then we decide on AVX optimisations based on that #define and also the platform defines. Signed-off-by: Harry Mallon <hjmallon@gmail.com> * Include <limits> where required by newer compilers (#893) * Include <limits> where required by newer compilers Signed-off-by: Cary Phillips <cary@ilm.com> * Removed redundant #include <limits> Signed-off-by: Cary Phillips <cary@ilm.com> * add buffer size validation to FastHuf decode Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> * prevent overflow in RgbaFile cachePadding Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> * Use size_t for DWA buffersize calculation (#901) * Use size_t for DWA buffersize calculation Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> * use Int64 instead of size_t for buffersize calculations Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> Signed-off-by: Cary Phillips <cary@ilm.com> * prevent overflows by using Int64 for all vars in DWA initialize (#903) Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> Signed-off-by: Cary Phillips <cary@ilm.com> * update tileoffset sanitycheck to handle ripmaps (#910) * update tileoffset sanitycheck to handle ripmaps Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> * slight reorganization Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> * slight reorganization Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> * remove extra if statement from validateStreamSize Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> Signed-off-by: Cary Phillips <cary@ilm.com> * additional verification of DWA data sizes (#914) Signed-off-by: Peter Hillman <peterh@wetafx.co.nz> * Release notes for v2.5.5 Signed-off-by: Cary Phillips <cary@ilm.com> * fix merge of ImfTiledInputFile.cpp Signed-off-by: Cary Phillips <cary@ilm.com> * Bump version for 2.5.5 Signed-off-by: Cary Phillips <cary@ilm.com> * Only wait for and join joinable threads (#921) Signed-off-by: Cary Phillips <cary@ilm.com> * Fixed botched merge or IlmThread.cpp/IlmThreadPool.cpp Signed-off-by: Cary Phillips <cary@ilm.com> * Fix 2.5.5 release date Signed-off-by: Cary Phillips <cary@ilm.com> Co-authored-by: Harry Mallon <hjmallon@gmail.com> Co-authored-by: Peter Hillman <peterh@wetafx.co.nz>
AcademySoftwareFoundation · Feb 12, 2021 · 4212416 · 4212416
1 parent a40a615
commit 4212416
Show file tree

Hide file tree

Showing 20 changed files with 244 additions and 87 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,5 +1,6 @@
 # OpenEXR Release Notes
 
+* [Version 2.5.5](#version-255-february-12-2021) February 12, 2021
 * [Version 2.5.4](#version-254-december-31-2020) December 31, 2020
 * [Version 2.5.3](#version-253-august-12-2020) August 12, 2020
 * [Version 2.5.2](#version-252-june-15-2020) June 15, 2020
@@ -38,8 +39,34 @@
 * [Version 1.0.1](#version-101)
 * [Version 1.0](#version-10)
 
-## Version 2.5.4 (December 31, 2020)
+## Version 2.5.5 (February 12, 2021)
+
+Patch release with various bug/sanitizer/security fixes, primarily
+related to reading corrupted input files, but also a fix for universal
+build support on macOS.
+
+Specific OSS-fuzz issues include:
 
+* OSS-fuzz [#30291](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=30291)
+* OSS-fuzz [#29106](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=29106)
+* OSS-fuzz [#28971](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28971)
+* OSS-fuzz [#29829](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=29829)
+* OSS-fuzz [#30121](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=30121)
+
+### Merged Pull Requests
+
+* [#914](https://github.com/AcademySoftwareFoundation/openexr/pull/914) additional verification of DWA data sizes
+* [#910](https://github.com/AcademySoftwareFoundation/openexr/pull/910) update tileoffset sanitycheck to handle ripmaps
+* [#903](https://github.com/AcademySoftwareFoundation/openexr/pull/903) prevent overflows by using Int64 for all vars in DWA initialize
+* [#901](https://github.com/AcademySoftwareFoundation/openexr/pull/901) Use size_t for DWA buffersize calculation
+* [#897](https://github.com/AcademySoftwareFoundation/openexr/pull/897) prevent overflow in RgbaFile cachePadding
+* [#896](https://github.com/AcademySoftwareFoundation/openexr/pull/896) add buffer size validation to FastHuf decode
+* [#893](https://github.com/AcademySoftwareFoundation/openexr/pull/893) Include <limits> where required by newer compilers
+* [#889](https://github.com/AcademySoftwareFoundation/openexr/pull/889) Add explicit #include <limits> for numeric_limits
+* [#854](https://github.com/AcademySoftwareFoundation/openexr/pull/854) Fix Apple Universal 2 (arm64/x86_64) builds
+
+## Version 2.5.4 (December 31, 2020)
+
 Patch release with various bug/sanitizer/security fixes, primarily
 related to reading corrupted input files.
 

diff --git a/IlmBase/IlmThread/IlmThread.cpp b/IlmBase/IlmThread/IlmThread.cpp
@@ -72,6 +72,18 @@ Thread::~Thread ()
         _thread.join ();
 }
 
+void
+Thread::join()
+{
+    if ( _thread.joinable () )
+        _thread.join ();
+}
+
+bool
+Thread::joinable() const
+{
+    return _thread.joinable();
+}
 
 void
 Thread::start ()
@@ -108,6 +120,19 @@ Thread::start ()
 {
     throw IEX_NAMESPACE::NoImplExc ("Threads not supported on this platform.");
 }
+
+void
+Thread::join () 
+{
+    throw IEX_NAMESPACE::NoImplExc ("Threads not supported / enabled on this platform.");
+}
+
+bool
+Thread::joinable () const
+{
+    throw IEX_NAMESPACE::NoImplExc ("Threads not supported / enabled on this platform.");
+}
+
 #   endif
 #endif
 

diff --git a/IlmBase/IlmThread/IlmThread.h b/IlmBase/IlmThread/IlmThread.h
@@ -129,6 +129,12 @@ class Thread
     ILMTHREAD_EXPORT void         start ();
     ILMTHREAD_EXPORT virtual void run () = 0;
 
+    //
+    // wait for thread to exit - must be called before deleting thread
+    //
+    void join();
+    bool joinable() const;
+
   private:
 
 #ifdef ILMBASE_FORCE_CXX03

diff --git a/IlmBase/IlmThread/IlmThreadPool.cpp b/IlmBase/IlmThread/IlmThreadPool.cpp
@@ -423,16 +423,24 @@ DefaultThreadPoolProvider::finish ()
     size_t curT = _data.threads.size();
     for (size_t i = 0; i != curT; ++i)
     {
-        _data.taskSemaphore.post();
-        _data.threadSemaphore.wait();
+        if (_data.threads[i]->joinable())
+        {
+            _data.taskSemaphore.post();
+            _data.threadSemaphore.wait();
+        }
     }
 
     //
     // Join all the threads
     //
     for (size_t i = 0; i != curT; ++i)
-        delete _data.threads[i];
+    {
+        if (_data.threads[i]->joinable())
+            _data.threads[i]->join();
 
+        delete _data.threads[i];
+    }
+
     Lock lock1 (_data.taskMutex);
 #ifdef ILMBASE_FORCE_CXX03
     Lock lock2 (_data.stopMutex);

diff --git a/IlmBase/configure.ac b/IlmBase/configure.ac
@@ -4,11 +4,11 @@ dnl Copyright Contributors to the OpenEXR Project.
 dnl
 
 dnl Process this file with autoconf to produce a configure script.
-AC_INIT(IlmBase, 2.5.4)
+AC_INIT(IlmBase, 2.5.5)
 
 AC_SUBST(ILMBASE_VERSION_MAJOR, 2)
 AC_SUBST(ILMBASE_VERSION_MINOR, 5)
-AC_SUBST(ILMBASE_VERSION_PATCH, 4)
+AC_SUBST(ILMBASE_VERSION_PATCH, 5)
 
 AC_SUBST(ILMBASE_VERSION, ${ILMBASE_VERSION_MAJOR}.${ILMBASE_VERSION_MINOR}.${ILMBASE_VERSION_PATCH})
 AC_SUBST(ILMBASE_VERSION_API, ${ILMBASE_VERSION_MAJOR}_${ILMBASE_VERSION_MINOR})
@@ -22,7 +22,7 @@ AM_MAINTAINER_MODE
 
 
 LIBTOOL_CURRENT=25
-LIBTOOL_REVISION=3
+LIBTOOL_REVISION=4
 LIBTOOL_AGE=0
 LIBTOOL_VERSION=$LIBTOOL_CURRENT:$LIBTOOL_REVISION:$LIBTOOL_AGE
 AC_SUBST(LIBTOOL_VERSION)

diff --git a/OpenEXR/IlmImf/ImfDeepScanLineInputFile.cpp b/OpenEXR/IlmImf/ImfDeepScanLineInputFile.cpp
@@ -65,12 +65,11 @@
 
 #include "Iex.h"
 
+#include <algorithm>
+#include <assert.h>
 #include <string>
 #include <vector>
-#include <assert.h>
 #include <limits>
-#include <algorithm>
-
 
 #include "ImfNamespace.h"
 OPENEXR_IMF_INTERNAL_NAMESPACE_SOURCE_ENTER

diff --git a/OpenEXR/IlmImf/ImfDeepTiledInputFile.cpp b/OpenEXR/IlmImf/ImfDeepTiledInputFile.cpp
@@ -64,6 +64,8 @@
 #include <vector>
 #include <algorithm>
 #include <assert.h>
+#include <string>
+#include <vector>
 #include <limits>
 
 #include "ImfNamespace.h"

diff --git a/OpenEXR/IlmImf/ImfDwaCompressor.cpp b/OpenEXR/IlmImf/ImfDwaCompressor.cpp
@@ -2606,6 +2606,14 @@ DwaCompressor::uncompress
             throw IEX_NAMESPACE::BaseExc("DC data corrupt.");
         }
     }
+    else
+    {
+        // if the compressed size is 0, then the uncompressed size must also be zero
+        if (totalDcUncompressedCount!=0)
+        {
+             throw IEX_NAMESPACE::BaseExc("DC data corrupt.");
+        }
+    }
 
     //
     // Uncompress the RLE data into _rleBuffer, then unRLE the results
@@ -2932,19 +2940,21 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
     // of channels we have. 
     //
 
-    int maxOutBufferSize  = 0;
-    int numLossyDctChans  = 0;
-    int unknownBufferSize = 0;
-    int rleBufferSize     = 0;
+    Int64 maxOutBufferSize  = 0;
+    Int64 numLossyDctChans  = 0;
+    Int64 unknownBufferSize = 0;
+    Int64 rleBufferSize     = 0;
 
-    int maxLossyDctAcSize = (int)ceil ((float)numScanLines() / 8.0f) * 
-                            (int)ceil ((float)(_max[0] - _min[0] + 1) / 8.0f) *
+    Int64 maxLossyDctAcSize = static_cast<Int64>(ceil ((float)numScanLines() / 8.0f)) *
+                            static_cast<Int64>(ceil ((float)(_max[0] - _min[0] + 1) / 8.0f)) *
                             63 * sizeof (unsigned short);
 
-    int maxLossyDctDcSize = (int)ceil ((float)numScanLines() / 8.0f) * 
-                            (int)ceil ((float)(_max[0] - _min[0] + 1) / 8.0f) *
+    Int64 maxLossyDctDcSize = static_cast<Int64>(ceil ((float)numScanLines() / 8.0f)) *
+                            static_cast<Int64>(ceil ((float)(_max[0] - _min[0] + 1) / 8.0f)) *
                             sizeof (unsigned short);
 
+    Int64 pixelCount = static_cast<Int64>(numScanLines()) * static_cast<Int64>(_max[0] - _min[0] + 1);
+
     for (unsigned int chan = 0; chan < _channelData.size(); ++chan)
     {
         switch (_channelData[chan].compression)
@@ -2959,8 +2969,8 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
             //
 
             maxOutBufferSize += std::max(
-                            (int)(2 * maxLossyDctAcSize + 65536),
-                            (int)compressBound (maxLossyDctAcSize) );
+                            2lu * maxLossyDctAcSize + 65536lu,
+                            static_cast<Int64>(compressBound (maxLossyDctAcSize)) );
             numLossyDctChans++;
             break;
 
@@ -2971,8 +2981,7 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
                 // of the source data.
                 //
 
-                int rleAmount = 2 * numScanLines() * (_max[0] - _min[0] + 1) *
-                                OPENEXR_IMF_NAMESPACE::pixelTypeSize (_channelData[chan].type);
+                Int64 rleAmount = 2 * pixelCount * OPENEXR_IMF_NAMESPACE::pixelTypeSize (_channelData[chan].type);
 
                 rleBufferSize += rleAmount;
             }
@@ -2981,8 +2990,7 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
 
           case UNKNOWN:
 
-            unknownBufferSize += numScanLines() * (_max[0] - _min[0] + 1) *
-                                 OPENEXR_IMF_NAMESPACE::pixelTypeSize (_channelData[chan].type);
+            unknownBufferSize += pixelCount * OPENEXR_IMF_NAMESPACE::pixelTypeSize (_channelData[chan].type);
             break;
 
           default:
@@ -2999,13 +3007,13 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
     // which could take slightly more space
     //
 
-    maxOutBufferSize += (int)compressBound ((uLongf)rleBufferSize);
+    maxOutBufferSize += static_cast<Int64>(compressBound (rleBufferSize));
 
     //
     // And the same goes for the UNKNOWN data
     //
 
-    maxOutBufferSize += (int)compressBound ((uLongf)unknownBufferSize);
+    maxOutBufferSize += static_cast<Int64>(compressBound (unknownBufferSize));
 
     //
     // Allocate a zip/deflate compressor big enought to hold the DC data
@@ -3052,7 +3060,7 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
     // to Huffman encoding
     //
 
-    if (static_cast<size_t>(maxLossyDctAcSize * numLossyDctChans) > _packedAcBufferSize)
+    if (maxLossyDctAcSize * numLossyDctChans > _packedAcBufferSize)
     {
         _packedAcBufferSize = maxLossyDctAcSize * numLossyDctChans;
         if (_packedAcBuffer != 0) 
@@ -3064,15 +3072,15 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
     // _packedDcBuffer holds one quantized DCT coef per 8x8 block
     //
 
-    if (static_cast<size_t>(maxLossyDctDcSize * numLossyDctChans) > _packedDcBufferSize)
+    if (maxLossyDctDcSize * numLossyDctChans > _packedDcBufferSize)
     {
         _packedDcBufferSize = maxLossyDctDcSize * numLossyDctChans;
         if (_packedDcBuffer != 0) 
             delete[] _packedDcBuffer;
         _packedDcBuffer     = new char[_packedDcBufferSize];
     }
 
-    if (static_cast<size_t>(rleBufferSize) > _rleBufferSize) 
+    if ( rleBufferSize > _rleBufferSize )
     {
         _rleBufferSize = rleBufferSize;
         if (_rleBuffer != 0) 
@@ -3091,7 +3099,7 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
     // all in one swoop (for each compression scheme).
     //
 
-    int planarUncBufferSize[NUM_COMPRESSOR_SCHEMES];
+    Int64 planarUncBufferSize[NUM_COMPRESSOR_SCHEMES];
     for (int i=0; i<NUM_COMPRESSOR_SCHEMES; ++i)
         planarUncBufferSize[i] = 0;
 
@@ -3103,14 +3111,12 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
             break;
 
           case RLE:
-            planarUncBufferSize[RLE] +=
-                     numScanLines() * (_max[0] - _min[0] + 1) *
+            planarUncBufferSize[RLE] += pixelCount *
                      OPENEXR_IMF_NAMESPACE::pixelTypeSize (_channelData[chan].type);
             break;
 
           case UNKNOWN: 
-            planarUncBufferSize[UNKNOWN] +=
-                     numScanLines() * (_max[0] - _min[0] + 1) *
+            planarUncBufferSize[UNKNOWN] += pixelCount *
                      OPENEXR_IMF_NAMESPACE::pixelTypeSize (_channelData[chan].type);
             break;
 
@@ -3127,17 +3133,23 @@ DwaCompressor::initializeBuffers (size_t &outBufferSize)
 
     if (planarUncBufferSize[UNKNOWN] > 0)
     {
-        planarUncBufferSize[UNKNOWN] = 
-            compressBound ((uLongf)planarUncBufferSize[UNKNOWN]);
+        planarUncBufferSize[UNKNOWN] =
+                static_cast<Int64>( compressBound (planarUncBufferSize[UNKNOWN]) );
     }
 
     for (int i = 0; i < NUM_COMPRESSOR_SCHEMES; ++i)
     {
-        if (static_cast<size_t>(planarUncBufferSize[i]) > _planarUncBufferSize[i]) 
+        if ( planarUncBufferSize[i] > _planarUncBufferSize[i])
         {
             _planarUncBufferSize[i] = planarUncBufferSize[i];
             if (_planarUncBuffer[i] != 0) 
                 delete[] _planarUncBuffer[i];
+
+            if (planarUncBufferSize[i] > std::numeric_limits<size_t>::max())
+            {
+                throw IEX_NAMESPACE::ArgExc("DWA buffers too large");
+            }
+
             _planarUncBuffer[i] = new char[planarUncBufferSize[i]];
         }
     }

diff --git a/OpenEXR/IlmImf/ImfDwaCompressor.h b/OpenEXR/IlmImf/ImfDwaCompressor.h
@@ -169,16 +169,16 @@ class DwaCompressor: public Compressor
     std::vector<CscChannelSet> _cscSets;
     std::vector<Classifier>    _channelRules;
 
-    char             *_packedAcBuffer;
-    size_t            _packedAcBufferSize;
-    char             *_packedDcBuffer;
-    size_t            _packedDcBufferSize;
-    char             *_rleBuffer;
-    size_t            _rleBufferSize;
-    char             *_outBuffer;
-    size_t            _outBufferSize;
-    char             *_planarUncBuffer[NUM_COMPRESSOR_SCHEMES];
-    size_t            _planarUncBufferSize[NUM_COMPRESSOR_SCHEMES];
+    char*             _packedAcBuffer;
+    Int64             _packedAcBufferSize;
+    char*             _packedDcBuffer;
+    Int64             _packedDcBufferSize;
+    char*             _rleBuffer;
+    Int64             _rleBufferSize;
+    char*             _outBuffer;
+    Int64             _outBufferSize;
+    char*             _planarUncBuffer[NUM_COMPRESSOR_SCHEMES];
+    Int64             _planarUncBufferSize[NUM_COMPRESSOR_SCHEMES];
 
     Zip              *_zip;
     float             _dwaCompressionLevel;