Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building problems by using add_subdirectory #1

Closed
Febbe opened this issue Jun 21, 2020 · 4 comments
Closed

Building problems by using add_subdirectory #1

Febbe opened this issue Jun 21, 2020 · 4 comments
Assignees
Labels
Component - Build CMake, Autotools Priority - 1. High 🔼 These are important issues that should be resolved in the next release Type - Bug Please report security issues to help@hdfgroup.org instead of creating an issue on GitHub

Comments

@Febbe
Copy link

Febbe commented Jun 21, 2020

It is not possible to build against hdf5 when the lib is added with add_subdirectory. The header file H5pubconf.h can not be found the include_dirs.
All generated header files with public visibility should be configured into an extra directory like ${CMAKE_CURRENT_BUILD_DIR}/generated/h5 and this dir should be included with

target_include_directories(mylib PUBLIC
  $<BUILD_INTERFACE:${CMAKE_CURRENT_BUILD_DIR}/generated>
  $<INSTALL_INTERFACE:include/mylib>  # <prefix>/include/mylib
)
seanm added a commit to seanm/hdf5 that referenced this issue Jun 29, 2020
…inator

Found by ASan:

==18515==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200001a791 at pc 0x00010111b47a bp 0x7ffeefbfe8f0 sp 0x7ffeefbfe098
READ of size 2 at 0x60200001a791 thread T0
    #0 0x10111b479 in wrap_strlen (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x14479)
    HDFGroup#1 0x100841c4c in H5T__conv_vlen H5Tconv.c:3193
    HDFGroup#2 0x1008113d9 in H5T_convert H5T.c:5024
    HDFGroup#3 0x100379fbb in H5D__scatgath_write H5Dscatgath.c:701
    HDFGroup#4 0x10032d235 in H5D__contig_write H5Dcontig.c:657
    HDFGroup#5 0x10036c768 in H5D__write H5Dio.c:819
    HDFGroup#6 0x10036b650 in H5Dwrite H5Dio.c:335
    HDFGroup#7 0x100115fa7 in H5::DataSet::write(void const*, H5::DataType const&, H5::DataSpace const&, H5::DataSpace const&, H5::DSetMemXferPropList const&) const H5DataSet.cpp:506
    HDFGroup#8 0x100031d9a in test_vlstrings tvlstr.cpp:191
    HDFGroup#9 0x1001aec42 in PerformTests testframe.c:323
    HDFGroup#10 0x100002ac0 in main testhdf5.cpp:116
    HDFGroup#11 0x7fff5eec6014 in start (libdyld.dylib:x86_64+0x1014)

0x60200001a791 is located 0 bytes to the right of 1-byte region [0x60200001a790,0x60200001a791)
allocated by thread T0 here:
    #0 0x10115c547 in wrap_calloc (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x55547)
    HDFGroup#1 0x100031d70 in test_vlstrings tvlstr.cpp:187
    HDFGroup#2 0x1001aec42 in PerformTests testframe.c:323
    HDFGroup#3 0x100002ac0 in main testhdf5.cpp:116
    HDFGroup#4 0x7fff5eec6014 in start (libdyld.dylib:x86_64+0x1014)
seanm added a commit to seanm/hdf5 that referenced this issue Jun 29, 2020
…H5Rdereference2()

Found by ASan:

==52756==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffeefbfedc8 at pc 0x000100be250f bp 0x7ffeefbfe890 sp 0x7ffeefbfe888
READ of size 1 at 0x7ffeefbfedc8 thread T0
    #0 0x100be250e in H5R__dereference H5Rint.c:416
    HDFGroup#1 0x100bddbd1 in H5Rdereference2 H5R.c:185
    HDFGroup#2 0x1001c168c in test_reference_region trefer.c:798
    HDFGroup#3 0x1001ba134 in test_reference trefer.c:1863
    HDFGroup#4 0x100660c42 in PerformTests testframe.c:323
    HDFGroup#5 0x100001fdc in main testhdf5.c:77
    HDFGroup#6 0x7fff5eec6014 in start (libdyld.dylib:x86_64+0x1014)

Address 0x7ffeefbfedc8 is located in stack of thread T0 at offset 616 in frame
    #0 0x1001be1df in test_reference_region trefer.c:507
seanm added a commit to seanm/hdf5 that referenced this issue Jun 29, 2020
… for the actual data

Found by ASan:

==19994==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffeefbfe920 at pc 0x0001010b064f bp 0x7ffeefbfc810 sp 0x7ffeefbfbfc0
READ of size 128 at 0x7ffeefbfe920 thread T0
    #0 0x1010b064e in __asan_memcpy (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x5264e)
    HDFGroup#1 0x1002cde61 in H5D__gather_mem H5Dscatgath.c:428
    HDFGroup#2 0x1002d0ca4 in H5D__scatgath_write H5Dscatgath.c:663
    HDFGroup#3 0x10025c948 in H5D__chunk_write H5Dchunk.c:2470
    HDFGroup#4 0x1002c3768 in H5D__write H5Dio.c:819
    HDFGroup#5 0x1002c2650 in H5Dwrite H5Dio.c:335
    HDFGroup#6 0x1000d4b8a in H5TBwrite_fields_index H5TB.c:729
    HDFGroup#7 0x100008c13 in test_table test_table.c:1420
    HDFGroup#8 0x1000028ff in main test_table.c:1686
    HDFGroup#9 0x7fff5eec6014 in start (libdyld.dylib:x86_64+0x1014)

Address 0x7ffeefbfe920 is located in stack of thread T0 at offset 4160 in frame
    #0 0x1000029cf in test_table test_table.c:211
soumagne pushed a commit that referenced this issue Sep 16, 2020
…h routine.

The test "driver_addr != sblock->driver_addr" is failing for superblock version 2 & 3.
Fix: there is no driver_addr in superblock version 2 & 3.
It should decode the root group object header address (root_addr) and verify accordingly.
soumagne pushed a commit that referenced this issue Sep 16, 2020
…vd_multi_memory_fix to multi

* commit '09981b58d800d784a2aa7e3ea5f7f1cad576e8db':
  Fixes a leak of the metadata index memory
@alephpiece
Copy link

Is it possible to use hdf5 targets in CMake? I didn't find a manual anywhere (for 1.10.5). I have to add include directories and link libraries to my target by myself because I don't know what target I could use.

If hdf5 is included as a subproject, there will be two targets, hdf5-shared and hdf5-static, but it seems that these targets marked their include directories as private. I have to extract those directories from hdf5 targets.

Is there a simple way to link hdf5 libraries in a CMake file?

find_package(HDF5 1.10.5 QUIET COMPONENTS C)
if (HDF5_FOUND)
    
    target_include_directories(my_target INTERFACE ${HDF5_INCLUDE_DIRS})
    target_compile_definitions(my_target INTERFACE ${HDF5_DEFINITIONS})
    target_link_libraries(my_target INTERFACE ${HDF5_LIBRARIES})

else ()

    config_hdf5()
    add_subdirectory(hdf5)

    target_include_directories(my_target INTERFACE $<TARGET_PROPERTY:hdf5-shared,INCLUDE_DIRECTORIES>)
    target_link_libraries(my_target INTERFACE hdf5-shared)
    add_dependencies(my_target hdf5-shared)

endif()

@tueda
Copy link

tueda commented Oct 4, 2020

While hdf5-1_12_0 doesn't work with the following CMakeLists.txt using add_subdirectory to add HDF5 (because of H5pubconf.h), the current development version (9df9061) seems to work (at least, for simple C examples):

cmake_minimum_required(VERSION 3.12)

project(test1 C)

set(HDF5_EXTERNALLY_CONFIGURED 1)
set(BUILD_TESTING OFF CACHE BOOL "Build HDF5 Unit Testing")
set(BUILD_SHARED_LIBS OFF CACHE BOOL "Build Shared Libraries")
set(HDF5_BUILD_CPP_LIB OFF CACHE BOOL "Build HDF5 C++ Library")
set(HDF5_BUILD_HL_LIB OFF CACHE BOOL "Build HIGH Level HDF5 Library")
set(HDF5_BUILD_TOOLS OFF CACHE BOOL "Build HDF5 Tools")
set(HDF5_BUILD_EXAMPLES OFF CACHE BOOL "Build HDF5 Library Examples")
add_subdirectory(extern/hdf5)

add_executable(test1 test1.c)
target_link_libraries(test1 PRIVATE hdf5-static)

A problem I found is that when I modify CMakeLists.txt all the C sources of hdf5-static are always recompiled, which takes a non-negligible time.

@tueda
Copy link

tueda commented Oct 5, 2020

For reference for those who hit this issue: I have realized there are related posts in the forum,

The former solved my problem (hdf5-static always recompiled).

@byrnHDF byrnHDF self-assigned this Feb 19, 2021
vchoi-hdfgroup added a commit that referenced this issue Apr 22, 2021
vchoi-hdfgroup pushed a commit that referenced this issue Jan 5, 2022
See Kent's documentation "Designed to Fail Tests and Issues".
(A) Fix for issue #1: HDassert the < and = cases between old and new
entry length.  John will take care of the > case.
(B) Fix for issue #3:  set the cache copy of page_size again if different
    from f->shared->fs_page_size.
@derobins derobins changed the title [cmake] building problems by using add_subdirectory Building problems by using add_subdirectory May 3, 2023
@derobins derobins added Priority - 1. High 🔼 These are important issues that should be resolved in the next release Component - Build CMake, Autotools Type - Bug Please report security issues to help@hdfgroup.org instead of creating an issue on GitHub labels May 3, 2023
derobins added a commit to derobins/hdf5 that referenced this issue Mar 21, 2024
Both H5Dchunk_iter() and H5Dget_chunk_info(_by_coord)() did not take
the size of the user block into account when reporting addresses. Since
the HDFGroup#1 use of these functions is to root around in the file for the raw
data, this is kind of a problem.

This is just the library changes. Tests will be in a future commit.
derobins added a commit to derobins/hdf5 that referenced this issue Mar 25, 2024
Both H5Dchunk_iter() and H5Dget_chunk_info(_by_coord)() did not take
the size of the user block into account when reporting addresses. Since
the HDFGroup#1 use of these functions is to root around in the file for the raw
data, this is kind of a problem.

Fixes GitHub issue HDFGroup#3003
derobins added a commit that referenced this issue Mar 26, 2024
Both H5Dchunk_iter() and H5Dget_chunk_info(_by_coord)() did not take
the size of the user block into account when reporting addresses. Since
the #1 use of these functions is to root around in the file for the raw
data, this is kind of a problem.

Fixes GitHub issue #3003
lrknox pushed a commit to lrknox/hdf5 that referenced this issue Mar 29, 2024
…#4236)

Both H5Dchunk_iter() and H5Dget_chunk_info(_by_coord)() did not take
the size of the user block into account when reporting addresses. Since
the HDFGroup#1 use of these functions is to root around in the file for the raw
data, this is kind of a problem.

Fixes GitHub issue HDFGroup#3003
lrknox added a commit that referenced this issue Mar 29, 2024
* Take user block into account when returning chunk addresses (#4236)

Both H5Dchunk_iter() and H5Dget_chunk_info(_by_coord)() did not take
the size of the user block into account when reporting addresses. Since
the #1 use of these functions is to root around in the file for the raw
data, this is kind of a problem.

Fixes GitHub issue #3003

* Fix a minor warning in h5test.c (#4242)

* Turn on -Werror for Java in GitHub -Werror workflows (#4243)

* Update Windows CI to not install ninja (#4230)

* Rework Fortran macros to use the proper code. (#4240)

* Correct reference copy for 16 API (#4244)

* Determine MPI LOGICAL during build, used in tests. (#4246)

* Skip userblock test in chunk_info.c for multi-file VFDs (#4249)

* Match generators with real cmake -G output on Windows (#4252)

* Add Julia GitHub Actions. (#4123)

* Re-revert to using autoreconf in autogen.sh (#4253)

We previously tried removing the per-tool invocation of the Autotools
and instead simply invoked autoreconf (PR #1906). This was reverted
when it turned out that the NAG Fortran compiler had trouble with an
undecorated -shared linker flag.

It turns out that this is due to a bug in libtool 2.4.2 and earlier.
Since this version of libtool is over a decade old, we're un-reverting
the change. We've added a release note for anyone who has to build
from source on elderly platforms.

Fixes #1343

* Rewrite H5T__path_find_real for clarity (#4225)

* Move conversion path free logic to helper function

* Add tgz extensions on names (#4255)

* Remove an error check regarding large cache objects (#4254)

* Remove an error check regarding large cache objects

In PR#4231 an assert() call was converted to a normal HDF5 error
check. It turns out that the original assert() was added by a
developer as a way of being alerted that large cache objects
existed instead of as a guard against incorrect behavior, making
it unnecessary in either debug or release builds.

The error check has been removed.

* Update RELEASE.txt

* File format security issues (#4234)

* Add job timeout to cygwin workflow (#4260)

* Replace user-define with user-defined (#4261)

* Improve the CMake clang -fsanitize=memory flags (#4267)

-fsanitize=memory is almost useless without
using -fsanitize-memory-track-origins=2 and we shoud probably add
-fno-optimize-sibling-calls as well.

* Add documentation (H5M) (#4259)

* Add documentation (H5P) (#4262)

* MPI type correction (#4268)

* corrected type for MPI_*_f2c APIs

* fixed return type of callback

* reset compilation flags of logical test program

* Clean up test/cmpd_dtransform.c (#4270)

* Clean up test/cmpd_dtransform.c

* Fix uninitialized memory warning from sanitizers
* FAIL_STACK_ERROR --> TEST_ERROR
* Emit output
* Delete test file when done

* Fix typo

* H5Fdelete() --> remove()

* Fix uninitialized memory issues in packet table (#4271)

* replace deprecated CMAKE_COMPILER_IS_GNU** (#4272)

* Prevent stack overflows in H5E__push_stack (#4264)

* Minor fixes after merge of file format security fixes (#4263)

* Update H5_IS_BUFFER_OVERFLOW to account for 'size' of 0

* Invert a few checks to avoid function call

* CHECK --> CHECK_PTR in tmisc.c (#4274)

* Add release note for CVE-2017-17507 (#4275)

* Update Cygwin installation guide (#4265)

* Addresses configuration fortran testing flags (#4276)

* turn warnings to errors in fortran configure test

* Intel fortran test fix

* Merge julia workflows into standard ci format (#4273)

* Fix range check in H5_addr_overlap (#4278)

When the H5_addr_overlap macro was updated to use H5_RANGE_OVERLAP,
it failed to take into account that H5_RANGE_OVERLAP expects the
range to be inclusive. This lead to an assertion failure in
H5MM_memcpy due to a memcpy operation on overlapping memory.
This has been fixed by subtracting 1 from the calculated high
bound values passed to H5_RANGE_OVERLAP

* Fix potential buffer read overflows in H5PB_read (#4279)

H5PB_read previously did not account for the fact that the size of the
read it's performing could overflow the page buffer pointer, depending
on the calculated offset for the read. This has been fixed by adjusting
the size of the read if it's determined that it would overflow the page.
@byrnHDF
Copy link
Contributor

byrnHDF commented May 22, 2024

Proof of working is at HDFGroup/hdf5-examples#49

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component - Build CMake, Autotools Priority - 1. High 🔼 These are important issues that should be resolved in the next release Type - Bug Please report security issues to help@hdfgroup.org instead of creating an issue on GitHub
Projects
None yet
Development

No branches or pull requests

5 participants