Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Libraries get linked in multiple times, RPATH is not properly set #1439

Closed
brycelelbach opened this issue Mar 31, 2015 · 7 comments
Closed

Libraries get linked in multiple times, RPATH is not properly set #1439

brycelelbach opened this issue Mar 31, 2015 · 7 comments
Assignees
@brycelelbach
Labels
affecting: LBNL category: CMake compiler: intel difficulty: medium platform: Xeon Phi
Milestone
0.9.11

Comments

@brycelelbach
Copy link
Member

When building for the Xeon Phi with Intel 15.0.1, I noticed that the overhauled CMake build system causes the same library to be linked in multiple times:

wash@bc1112-mic0:~/development/hpx-latest/BUILD_MIC$ ldd bin/hello_world 
    linux-vdso.so.1 =>  (0x00007fff9d3ff000)
    libhpx.so.0 => /global/u1/w/wash/development/hpx-latest/BUILD_MIC/lib/libhpx.so.0 (0x00002adcabacf000)
    libhpx_serialization.so.0 => /global/u1/w/wash/development/hpx-latest/BUILD_MIC/lib/libhpx_serialization.so.0 (0x00002adcad1c4000)
    libiostreams.so.0 => /global/u1/w/wash/development/hpx-latest/BUILD_MIC/lib/libiostreams.so.0 (0x00002adcad3d8000)
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libboost_chrono.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_chrono.so.1.55.0 (0x00002adcad70f000)
    libboost_date_time.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_date_time.so.1.55.0 (0x00002adcad918000)
    libboost_filesystem.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_filesystem.so.1.55.0 (0x00002adcadb2b000)
    libboost_program_options.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_program_options.so.1.55.0 (0x00002adcadd4d000)
    libboost_regex.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_regex.so.1.55.0 (0x00002adcadffa000)
    libboost_serialization.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_serialization.so.1.55.0 (0x00002adcae3f3000)
    libboost_system.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_system.so.1.55.0 (0x00002adcae6a3000)
    libboost_thread.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_thread.so.1.55.0 (0x00002adcae8a7000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00002adcaeacb000)
    libboost_atomic.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_atomic.so.1.55.0 (0x00002adcaece9000)
    libtbbmalloc.so.2 => not found
    libtbbmalloc_proxy.so.2 => not found
    libhwloc.so.5 => /global/homes/w/wash/development/hwloc-1.7.2/MIC_INSTALL/lib/libhwloc.so.5 (0x00002adcaeeec000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00002adcaf12f000)
    librt.so.1 => /lib64/librt.so.1 (0x00002adcaf333000)
    libm.so.6 => /lib64/libm.so.6 (0x00002adcaf53c000)
    libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002adcaf76b000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002adcafa65000)
    libc.so.6 => /lib64/libc.so.6 (0x00002adcafc78000)
    /lib64/ld-linux-k1om.so.2 (0x00002adcab8ae000)
    libtbbmalloc.so.2 => not found
    libtbbmalloc_proxy.so.2 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libtbbmalloc.so.2 => not found
    libtbbmalloc_proxy.so.2 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libtbbmalloc.so.2 => not found
    libtbbmalloc_proxy.so.2 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found

This may be an Intel specific issue - libintlc, libirng libimf and libsvml are all Intel-specific libraries. However, the multiple links to libtbbmalloc makes me think this may be a larger issue.

Additionally, even though I specified -DHPX_FULL_RPATH, the rpath for tbbmalloc has not been added.
@brycelelbach brycelelbach self-assigned this Mar 31, 2015
@brycelelbach
Copy link
Member Author

Okay, the multiple links appear to be a red herring; they go away if the correct RPATH is set or LD_LIBRARY_PATH is used. For example, using a (much) older version of HPX that correct RPATHs tbbmalloc, we get:

wash@bc1112-mic0:~/development/hpx/BUILD_MIC$ ldd bin/hello_world 
    linux-vdso.so.1 =>  (0x00007fff8dabe000)
    libiostreams.so.0 => /global/homes/w/wash/development/hpx/MIC_INSTALL/lib/hpx/libiostreams.so.0 (0x00002b3cbbd74000)
    libhpx.so.0 => /global/homes/w/wash/development/hpx/MIC_INSTALL/lib/hpx/libhpx.so.0 (0x00002b3cbc140000)
    libhpx_serialization.so.0 => /global/homes/w/wash/development/hpx/MIC_INSTALL/lib/hpx/libhpx_serialization.so.0 (0x00002b3cbe13f000)
    libboost_date_time.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_date_time.so.1.55.0 (0x00002b3cbe359000)
    libboost_filesystem.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_filesystem.so.1.55.0 (0x00002b3cbe56d000)
    libboost_program_options.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_program_options.so.1.55.0 (0x00002b3cbe78e000)
    libboost_regex.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_regex.so.1.55.0 (0x00002b3cbea3c000)
    libboost_serialization.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_serialization.so.1.55.0 (0x00002b3cbee34000)
    libboost_system.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_system.so.1.55.0 (0x00002b3cbf0e4000)
    libboost_thread.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_thread.so.1.55.0 (0x00002b3cbf2e9000)
    libboost_atomic.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_atomic.so.1.55.0 (0x00002b3cbf50d000)
    libboost_chrono.so.1.55.0 => /global/homes/w/wash/development/boost_1_55_0/MIC_INSTALL/lib/libboost_chrono.so.1.55.0 (0x00002b3cbf70f000)
    libtbbmalloc.so.2 => /global/babbage/nsg/opt/intel/composerxe/composer_xe_2013.4.183/composer_xe_2013.4.183/tbb/lib/mic/libtbbmalloc.so.2 (0x00002b3cbf919000)
    libtbbmalloc_proxy.so.2 => /global/babbage/nsg/opt/intel/composerxe/composer_xe_2013.4.183/composer_xe_2013.4.183/tbb/lib/mic/libtbbmalloc_proxy.so.2 (0x00002b3cbfb51000)
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libhwloc.so.5 => /global/homes/w/wash/development/hwloc-1.7.2/MIC_INSTALL/lib/libhwloc.so.5 (0x00002b3cbfd55000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00002b3cbff98000)
    librt.so.1 => /lib64/librt.so.1 (0x00002b3cc019c000)
    libm.so.6 => /lib64/libm.so.6 (0x00002b3cc03a5000)
    libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002b3cc05d4000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b3cc08ce000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b3cc0ae1000)
    libc.so.6 => /lib64/libc.so.6 (0x00002b3cc0cfe000)
    /lib64/ld-linux-k1om.so.2 (0x00002b3cbbb53000)
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found
    libimf.so => not found
    libsvml.so => not found
    libirng.so => not found
    libintlc.so.5 => not found

Note the libtbbmalloc link; there's just one now.

@hkaiser
Copy link
Member

hkaiser commented Mar 31, 2015

@brycelelbach: so can this be closed now? I'm confused.

@brycelelbach
Copy link
Member Author

Not yet, there's still the RPATH issue. I think the solution to this is for me to just add CMAKE_INSTALL_RPATH_USE_LINK_PATH to my build. I'm testing that now. If that works, we may want to make HPX_FULL_RPATH=ON set CMAKE_INSTALL_RPATH_USE_LINK_PATH.

@hkaiser
Copy link
Member

hkaiser commented Mar 31, 2015

Could this be related to #1299?

@hkaiser hkaiser added this to the 0.9.11 milestone Mar 31, 2015
@brycelelbach
Copy link
Member Author

I don't believe so. This issue is resolved by setting CMAKE_INSTALL_RPATH_USE_LINK_PATH.

@sithhell
Copy link
Member

sithhell commented Mar 31, 2015 via email

@hkaiser
Copy link
Member

hkaiser commented Mar 31, 2015

Is there anything we need to do to set CMAKE_INSTALL_RPATH_USE_LINK_PATH? Or is that something the user has to do - should it be documented, then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brycelelbach brycelelbach

Labels
affecting: LBNL category: CMake compiler: intel difficulty: medium platform: Xeon Phi
Projects
None yet
Milestone
0.9.11
Development

No branches or pull requests
3 participants
@sithhell @brycelelbach @hkaiser