Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) #405

bmribler · 2021-03-01T16:18:53Z

Description
Checked against buffer size to prevent segfault, in case of data corruption.

+ HDFFV-11159 CVE-2018-14033 (same issue as CVE-2020-10811) Buffer over-read in H5O_layout_decode
+ HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode

Platforms tested:
Linux/64 (jelly)

Description Checked against buffer size to prevent segfault, in case of data corruption. + HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode + HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode Platforms tested: Linux/64 (jelly)

after a more correct fix was applied, that is the check now accounted for the previous advance of the buffer pointer. Removed it.

tCVE_2018_11206_fill_old.h5 tCVE_2018_11206_fill_new.h5

This reverts commit c21324d.

This reverts commit 5832a70.

This reverts commit b45de82.

This reverts commit 1671982.

byrnHDF · 2021-03-16T12:41:02Z

tools/test/h5dump/CMakeTests.cmake

@@ -1172,6 +1174,10 @@
  # test to verify HDFFV-9407: long double full precision
  ADD_H5_GREP_TEST (t128bit_float 1 "1.123456789012345" -m %.35Lf t128bit_float.h5)

+  # test to verify HDFFV-10480: out of bounds read in H5O_fill_new[old]_decode
+  ADD_H5_TEST (tCVE_2018_11206_fill_old 1 tCVE_2018_11206_fill_old.h5)


I wonder if ADD_H5_TEST should be ADD_H5_GREP_TEST and search that the test failure that is expected and doesn't fail for some other reason.

I'm not sure, but TOOLTEST_FAIL took care of that, it would detect the segfault. I wish the author would put some comments to these macros... :-/

Oh it segfaults - okay then this is fine.

This reverts commit b343d66.

This reverts commit b8a0f9a.

lrknox · 2021-03-17T14:08:32Z

tools/test/h5dump/testh5dump.sh.in

+    # Segfault occurred
+    if [ $RET == 139 ] ; then
+        nerrors="`expr $nerrors + 1`"
+        echo " FAILED"


I think it might be worthwhile to differentiate the two "FAILED" messages to make the log more meaningful and also to make clear in the second case that an expected h5dump failure didn't happen. Also note that all other FAILED messages in the script are surrounded by *s: FAILED. If you follow that convention it makes it easier to distinguish failed tests in the logs from incidental occurrences of "FAILED" that don't indicate test failures.

I suggest:
line 891 "FAILED - [h5dump] test failed with segmentation fault"
line 895 "FAILED - [h5dump] test did not fail as expected"

[ ] - optional - could also be test name if available.

Ah, thank you! I knew about '' but must have used the existing PASSED line and changed to FAIL and forgot about the ''.

I've made the changes and resolved the conflict.

* Snapshot version 1.12 release 1-3. Update version to 1.12.1-4. * First cut of the H5 public API documentation. (#80) * First cut of the H5 public API documentation. * Added H5Z "bonus track." * Applied Quincey's patch. * Added the missing patches from Quincey's original patch. * H5PL (complete) and basic H5VL API documentation. * Added H5I API docs. * Added H5L API docs. * First installment from Elena's H5T batch. * Second installment of Elena's H5T batch. * Final installment of Elena's H5T batch. * Full set of current H5F documentation. (#105) * First cut of the H5 public API documentation. * Added H5Z "bonus track." * Applied Quincey's patch. * Added the missing patches from Quincey's original patch. * H5PL (complete) and basic H5VL API documentation. * Added H5I API docs. * Added H5L API docs. * First installment from Elena's H5T batch. * Second installment of Elena's H5T batch. * Final installment of Elena's H5T batch. * Migrated documentation for SWMR functions. * Catching up on MDC functions. * Integrated the H5F MDC function documentation. * Added MDC and parallel H5F functions. * Slightly updated main page. * Added doxygen/dox/H5AC_cache_config_t.dox to MANIFEST. * Doxygen - added (mostly) beginner functions (#112) * Doxygen - added (mostly) beginner functions * Removed duplicate H5Pset_szip function * Add src/H5module.h to MANIFEST. * close #195. (#196) * Update HDF5PluginMacros.cmake * Update HDF5PluginMacros.cmake * Avoid aligned access for references by decoding into temporary buffer and then copying the result into the actual buffer. Update test to be more thorough with using compound datatype fields everywhere. (#206) * Modify temporary rpath for testing in java example scripts. (#230) * Fix undefined left shifting of negative numbers (#338) Undefined Bahavior Sanitizer errored here about left shifting negative numbers. * Fixes various warnings noticed on Windows (#425) * Fixes various warnings noticed on Windows - Adds a prototype for our implementation of vasprintf - Return type of H5_get_utf16_str() is now non-const - Fixes possible uninitialized return type in Wremove_utf8 - Better isolation of fork() code in accum.c:test_swmr_write_big() - Better isolation of non-zlib code in dsets.c:test_filter_delete() - Removed unused variable in trefer.c:test_reference_cmpnd_obj() * Fixes clang-format issues * Applied clang-tidy readability-non-const-parameter warning fixes auto… (#429) * Automatically applied clang-tidy readability-avoid-const-params-in-decls fixes Removes useless const declarations. * Fixed most readability-non-const-parameter warnings These changes were made automatically by clang-tidy, but I manually reverted the changes related to the H5Z_func_t signature. * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Added C++11 override keyword where appropriate (#433) Added H5_OVERRIDE macro for compatibility with both C++11 and older. * Various clang tidy warning fixes (#448) * Fixed clang-tidy bugprone-reserved-identifier warnings * Fixed clang-tidy bugprone-assert-side-effect warnings * Fixed clang-tidy bugprone-copy-constructor-init warning * Fixed clang-tidy readability-redundant-preprocessor warning For error_test.c the removed code was already dead, because it was in the else of an `#if H5_USE_16_API` block. Based on H5Location.h, I think p_get_ref_obj_type was meant to be in `#ifndef DOXYGEN_SHOULD_SKIP_THIS` and an `#endif` was missing. Similarly, in the header, getObjTypeByIdx is only in H5_NO_DEPRECATED_SYMBOLS, not DOXYGEN_SHOULD_SKIP_THIS. * Fixed clang-tidy readability-redundant-string-init warnings * Fixed some clang-tidy performance-type-promotion-in-math-fn warnings * Fixed clang-tidy performance-unnecessary-value-param warnings * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Removed checks/workarounds for pre-C++89 compatibility (#449) After 30+ years, just assume that the following exist: - extension-less includes - namespaces - std:: - static_cast - bool * Fixed all clang-tidy bugprone-suspicious-string-compare warnings (#451) * Fixed all clang-tidy bugprone-suspicious-string-compare warnings This change was generated entirely by clang-tidy itself. * Reformat code with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Remove 2 functions incorrectly merged from develop in a cherry-pick merge of PR #451. * Purge the buffer used in type conversion. (#263) Some of the uniniitialized bits in the buffer may get carried through all the way to disk, creating a risk for leaks. We observed an msan error during the floating point output conversion. Due to the encoding certain bits could remain untouched during the conversion. In this draft we zero initialize the dbuf used by every convertor. * Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) (#405) * Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) Description Checked against buffer size to prevent segfault, in case of data corruption. + HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode + HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode Platforms tested: Linux/64 (jelly) * Accidentally left in another occurrence of the previous patch from user after a more correct fix was applied, that is the check now accounted for the previous advance of the buffer pointer. Removed it. * Typo * Fixed format issues. * Added test. * Changed arguments to ADD_H5_TEST * Fixing arguments to ADD_H5_TEST again. * Fixing arguments again. * Took out the CMake changes until Allen can help. * Added files: tCVE_2018_11206_fill_old.h5 tCVE_2018_11206_fill_new.h5 * Revert "Took out the CMake changes until Allen can help." This reverts commit c21324d. * Revert "Fixing arguments again." This reverts commit 5832a70. * Revert "Fixing arguments to ADD_H5_TEST again." This reverts commit b45de82. * Revert "Changed arguments to ADD_H5_TEST" This reverts commit 1671982. * Added first argument to ADD_H5_TEST for HDFFV-10480 fix. * Changed argument 0 to 1 * Revert "Changed argument 0 to 1" This reverts commit b343d66. * Revert "Added first argument to ADD_H5_TEST for HDFFV-10480 fix." This reverts commit b8a0f9a. * Added first argument and corrected the second. * Updated fixes for HDFFV-10480 and HDFFV-11159/HDFFV-11049 * Improved error messages. * Added description of the current HDF5 branches; added a draft of contribution policy. (#445) * Added description of the current HDF5 branches. * Removed capitalization in from Develop, Release, Feature to reflect the real naming schema * Added a draft of contributions guidance document. * Fixed typos. * Fixed section title. * Fixed typo. * Fixed typos and formatting. * Fixed many typos and simplified the text (e.g., removed testing instructions and left pointers to where to find them, etc.) * Fixed a typo. Please enter the commit message for your changes. Lines starting * Added contributing.md file and rearranged doc entries in alphabetical order. * ddressed Gerd's review comments; found and fixed more typos. * Addressed comments from Larry and Scot. * Fix CMake error message location. (#478) Print error message if Perl is not found. * Committing clang-format changes Co-authored-by: Gerd Heber <gheber@hdfgroup.org> Co-authored-by: bljhdf <58825073+bljhdf@users.noreply.github.com> Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org> Co-authored-by: Quincey Koziol <quincey@koziol.cc> Co-authored-by: Sean McBride <sean@rogue-research.com> Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com> Co-authored-by: Yu Feng <rainwoodman@gmail.com> Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com> Co-authored-by: epourmal <epourmal@hdfgroup.org> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* close #195. (#196) * Update HDF5PluginMacros.cmake * Update HDF5PluginMacros.cmake * Modify temporary rpath for testing in java example scripts. (#230) * Fix undefined left shifting of negative numbers (#338) Undefined Bahavior Sanitizer errored here about left shifting negative numbers. * Update license url (#332) * Modify temporary rpath for testing in java example scripts. * Update URL in source file Copyright headers for web copy of COPYING file - src and test directories. * Cleans up a couple of MSVC warnings in testhdf5 (#475) * Fixes a few testhdf5 warnings raised in Visual Studio Visual Studio is grumpier about treating pointers like integers than gcc. * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Fix typos and grammar errors. (#476) * Suppresses the tcheck_version test's abort dialog on Windows (#477) * Suppresses the tcheck_version test's abort dialog on Windows Windows raises a modal abort/retry/ignore dialog box when CRT calls abort(). This change installs a report hook that suppresses the dialog so that the CMake tests don't time out waiting for a nonexistent user to click a dialog box. * Committing clang-format changes * Removes __cdecl from callback Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Purge the buffer used in type conversion. (#263) Some of the uniniitialized bits in the buffer may get carried through all the way to disk, creating a risk for leaks. We observed an msan error during the floating point output conversion. Due to the encoding certain bits could remain untouched during the conversion. In this draft we zero initialize the dbuf used by every convertor. * Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) (#405) * Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) Description Checked against buffer size to prevent segfault, in case of data corruption. + HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode + HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode Platforms tested: Linux/64 (jelly) * Accidentally left in another occurrence of the previous patch from user after a more correct fix was applied, that is the check now accounted for the previous advance of the buffer pointer. Removed it. * Typo * Fixed format issues. * Added test. * Changed arguments to ADD_H5_TEST * Fixing arguments to ADD_H5_TEST again. * Fixing arguments again. * Took out the CMake changes until Allen can help. * Added files: tCVE_2018_11206_fill_old.h5 tCVE_2018_11206_fill_new.h5 * Revert "Took out the CMake changes until Allen can help." This reverts commit c21324d. * Revert "Fixing arguments again." This reverts commit 5832a70. * Revert "Fixing arguments to ADD_H5_TEST again." This reverts commit b45de82. * Revert "Changed arguments to ADD_H5_TEST" This reverts commit 1671982. * Added first argument to ADD_H5_TEST for HDFFV-10480 fix. * Changed argument 0 to 1 * Revert "Changed argument 0 to 1" This reverts commit b343d66. * Revert "Added first argument to ADD_H5_TEST for HDFFV-10480 fix." This reverts commit b8a0f9a. * Added first argument and corrected the second. * Updated fixes for HDFFV-10480 and HDFFV-11159/HDFFV-11049 * Improved error messages. * Added description of the current HDF5 branches; added a draft of contribution policy. (#445) * Added description of the current HDF5 branches. * Removed capitalization in from Develop, Release, Feature to reflect the real naming schema * Added a draft of contributions guidance document. * Fixed typos. * Fixed section title. * Fixed typo. * Fixed typos and formatting. * Fixed many typos and simplified the text (e.g., removed testing instructions and left pointers to where to find them, etc.) * Fixed a typo. Please enter the commit message for your changes. Lines starting * Added contributing.md file and rearranged doc entries in alphabetical order. * ddressed Gerd's review comments; found and fixed more typos. * Addressed comments from Larry and Scot. * Fix CMake error message location. (#478) Print error message if Perl is not found. * Committing clang-format changes * Update MANIFEST. Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org> Co-authored-by: Sean McBride <sean@rogue-research.com> Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Yu Feng <rainwoodman@gmail.com> Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com> Co-authored-by: epourmal <epourmal@hdfgroup.org>

bmribler requested review from derobins, fortnern, jhendersonHDF, jrmainzer, qkoziol, soumagne and vchoi-hdfgroup as code owners March 1, 2021 16:18

bmribler added 3 commits March 1, 2021 10:38

Accidentally left in another occurrence of the previous patch from user

c1bdb0c

after a more correct fix was applied, that is the check now accounted for the previous advance of the buffer pointer. Removed it.

Typo

c24a0c2

Fixed format issues.

1dec6d8

derobins approved these changes Mar 3, 2021

View reviewed changes

qkoziol approved these changes Mar 4, 2021

View reviewed changes

byrnHDF added the test needed label Mar 10, 2021

Added test.

a147ae6

bmribler requested a review from byrnHDF as a code owner March 15, 2021 20:36

bmribler added 5 commits March 15, 2021 23:31

Changed arguments to ADD_H5_TEST

1671982

Fixing arguments to ADD_H5_TEST again.

b45de82

Fixing arguments again.

5832a70

Took out the CMake changes until Allen can help.

c21324d

Added files:

8e71d59

tCVE_2018_11206_fill_old.h5 tCVE_2018_11206_fill_new.h5

bmribler requested a review from lrknox as a code owner March 16, 2021 05:23

bmribler added 6 commits March 16, 2021 01:26

Revert "Took out the CMake changes until Allen can help."

d0a00d1

This reverts commit c21324d.

Revert "Fixing arguments again."

0c7c02e

This reverts commit 5832a70.

Revert "Fixing arguments to ADD_H5_TEST again."

a030e91

This reverts commit b45de82.

Revert "Changed arguments to ADD_H5_TEST"

9ae8bd2

This reverts commit 1671982.

Added first argument to ADD_H5_TEST for HDFFV-10480 fix.

b8a0f9a

Changed argument 0 to 1

b343d66

byrnHDF reviewed Mar 16, 2021

View reviewed changes

Revert "Changed argument 0 to 1"

5060450

This reverts commit b343d66.

bmribler added 3 commits March 16, 2021 09:42

Revert "Added first argument to ADD_H5_TEST for HDFFV-10480 fix."

fcfbacb

This reverts commit b8a0f9a.

Added first argument and corrected the second.

5213c5a

Updated fixes for HDFFV-10480 and HDFFV-11159/HDFFV-11049

e5e2404

bmribler requested a review from bljhdf as a code owner March 16, 2021 17:21

byrnHDF approved these changes Mar 17, 2021

View reviewed changes

byrnHDF removed the test needed label Mar 17, 2021

lrknox requested changes Mar 17, 2021

View reviewed changes

bmribler and others added 2 commits March 17, 2021 12:42

Improved error messages.

0c04d8b

Merge branch 'develop' into bmr_dev_HDFFV-10480_HDFFV-11159

64490c6

lrknox approved these changes Mar 17, 2021

View reviewed changes

lrknox merged commit dafc728 into HDFGroup:develop Mar 19, 2021

This was referenced Apr 1, 2021

Hdf5 merge issue 488 v112 #534

Merged

Merge changes from PRs #263, 405, 445, 478 to support branches. #488

Closed

Hdf5 merge issue 488 v110 #535

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) #405

Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) #405

bmribler commented Mar 1, 2021 •

edited

Loading

byrnHDF Mar 16, 2021

bmribler Mar 16, 2021

byrnHDF Mar 17, 2021

lrknox Mar 17, 2021

bmribler Mar 17, 2021

bmribler Mar 17, 2021

Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) #405

Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) #405

Conversation

bmribler commented Mar 1, 2021 • edited Loading

byrnHDF Mar 16, 2021

Choose a reason for hiding this comment

bmribler Mar 16, 2021

Choose a reason for hiding this comment

byrnHDF Mar 17, 2021

Choose a reason for hiding this comment

lrknox Mar 17, 2021

Choose a reason for hiding this comment

bmribler Mar 17, 2021

Choose a reason for hiding this comment

bmribler Mar 17, 2021

Choose a reason for hiding this comment

bmribler commented Mar 1, 2021 •

edited

Loading