Linking issue with Boost

[akshayc11]

Hi,
When trying to compile the source code, I get the following errors:

make
[ 49%] Built target pagmo_static
Linking CXX executable main
src/libpagmo.a(jde.cpp.o): In function `void pagmo::rng_uint32::load<boost::archive::text_iarchive>(boost::archive::text_iarchive&, unsigned int) [clone .isra.324]':
jde.cpp:(.text+0x3e): undefined reference to `boost::archive::text_iarchive_impl<boost::archive::text_iarchive>::load(std::string&)'
src/libpagmo.a(jde.cpp.o): In function `void pagmo::rng_double::load<boost::archive::binary_iarchive>(boost::archive::binary_iarchive&, unsigned int) [clone .isra.348]':
jde.cpp:(.text+0x10f): undefined reference to `boost::archive::basic_binary_iprimitive<boost::archive::binary_iarchive, char, std::char_traits<char> >::load(std::string&)'
src/libpagmo.a(jde.cpp.o): In function `void pagmo::rng_double::load<boost::archive::text_iarchive>(boost::archive::text_iarchive&, unsigned int) [clone .isra.316]':
jde.cpp:(.text+0x4fe): undefined reference to `boost::archive::text_iarchive_impl<boost::archive::text_iarchive>::load(std::string&)'
src/libpagmo.a(jde.cpp.o): In function `void pagmo::rng_uint32::load<boost::archive::binary_iarchive>(boost::archive::binary_iarchive&, unsigned int) [clone .isra.356]':
jde.cpp:(.text+0x9ef): undefined reference to `boost::archive::basic_binary_iprimitive<boost::archive::binary_iarchive, char, std::char_traits<char> >::load(std::string&)'
src/libpagmo.a(jde.cpp.o): In function `boost::archive::basic_binary_oprimitive<boost::archive::binary_oarchive, char, std::char_traits<char> >::save_binary(void const*, unsigned long) [clone .isr\
a.162]':
... (a whole bunch of them)

I tried reinstalling boost to the latest version, and even that did not seem to work.
I performed the default:
.```
/bootstrap.sh
./b2
sudo ./b2 install

However, I still get the same issue. The paths to the boost libraries in the ccmake stage seem to be reasonable. So, I am not sure what is happening.

Please let me know if I am doing something wrong.

[darioizzo]

I would need more info to help. Its seems in your system the serialization library from boost is not found by the linker.

It may be related to this:

#149

If not, I need to know the output from your cmake, your system, etc. etc.

[akshayc11]

Hi,

I realized that if I changed the ccmake options to not build main, and build pyGMO only, I faced no issues during installation.
However, when I went with the default (with build main enabled), and making the modification (1) suggested in #149 the number of errors dropped to one error:

System Info:
Hardware:
processor       : 31
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz
stepping        : 7
microcode       : 0x70d
cpu MHz         : 1200.000
cache size      : 20480 KB
physical id     : 1
siblings        : 16
core id         : 7
cpu cores       : 8
apicid          : 47
initial apicid  : 47
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 5787.16
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual

OS: Ubuntu 14.04 64bit
gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.1) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Boost: 1.6.1 (Compiled and installed: I think even all the threads)

CMake results:
speech@gpu04:/data-local/akshayc/Workspace/Software/pagmo/build$ cmake ../
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- OS detected: Linux
-- CXX Compiler detected: GNU
-- CMake additional search path for libraries:
-- Performing Test GNUCXX_NO_STRICT_ALIASING
-- Performing Test GNUCXX_NO_STRICT_ALIASING - Failed
-- Performing Test GNUCXX_W_ALL
-- Performing Test GNUCXX_W_ALL - Success
-- Enabling '-Wall' compiler flag.
-- Performing Test GNUCXX_W_NODEPRECATED
-- Performing Test GNUCXX_W_NODEPRECATED - Success
-- Enabling '-Wno-deprecated' compiler flag.
-- Performing Test GNUCXX_W_EXTRA
-- Performing Test GNUCXX_W_EXTRA - Success
-- Enabling '-Wextra' compiler flag.
-- Performing Test GNUCXX_W_NOEXCEPT
-- Performing Test GNUCXX_W_NOEXCEPT - Success
-- Enabling '-Wnoexcept' compiler flag.
-- Performing Test GNUCXX_W_DISOPT
-- Performing Test GNUCXX_W_DISOPT - Success
-- Enabling '-Wdisabled-optimization' compiler flag.
-- Performing Test GNUCXX_FASTMATH
-- Performing Test GNUCXX_FASTMATH - Success
-- Enabling '-ffast-math' compiler flag.
-- Performing Test ALL_C11
-- Performing Test ALL_C11 - Success
-- Enabling '-std=c++11' compiler flag
-- CXX compilation flags:  -Wall -Wno-deprecated -Wextra -Wnoexcept -Wdisabled-optimization -ffast-math -std=c++11
-- Required Boost libraries: system;serialization;thread
-- Boost version: 1.61.0
-- Found the following Boost libraries:
--   system
--   serialization
--   thread
-- Detected Boost version: 106100
-- Boost include dirs: /usr/local/include
-- Boost libraries: /usr/local/lib/libboost_system-mt.so;/usr/local/lib/libboost_serialization-mt.so;/usr/local/lib/libboost_thread-mt.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Thread library: -lpthread
-- Build flags:  -Wall -Wno-deprecated -Wextra -Wnoexcept -Wdisabled-optimization -ffast-math -std=c++11 -pthread -DBOOST_THREAD_USE_DLL -DBOOST_SERIALIZATION_DYN_LINK=1
-- Module linker flags:
-- Shared linker flags:
-- Configuring done
-- Generating done
-- Build files have been written to: /data-local/akshayc/Workspace/Software/pagmo/build

On Running Make:

speech@gpu04:/data-local/akshayc/Workspace/Software/pagmo/build$ make
Scanning dependencies of target pagmo_static
[  1%] Building CXX object src/CMakeFiles/pagmo_static.dir/algorithm/base.cpp.o
[  2%] Building CXX object src/CMakeFiles/pagmo_static.dir/archipelago.cpp.o
[  2%] Building CXX object src/CMakeFiles/pagmo_static.dir/base_island.cpp.o
[  3%] Building CXX object src/CMakeFiles/pagmo_static.dir/island.cpp.o
[  4%] Building CXX object src/CMakeFiles/pagmo_static.dir/population.cpp.o
[  5%] Building CXX object src/CMakeFiles/pagmo_static.dir/algorithm/sea.cpp.o
[  5%] Building CXX object src/CMakeFiles/pagmo_static.dir/algorithm/de.cpp.o
...
[ 98%] Building CXX object src/CMakeFiles/pagmo_static.dir/util/neighbourhood.cpp.o
[ 99%] Building CXX object src/CMakeFiles/pagmo_static.dir/util/race_pop.cpp.o
[100%] Building CXX object src/CMakeFiles/pagmo_static.dir/util/race_algo.cpp.o
Linking CXX static library libpagmo.a
[100%] Built target pagmo_static
Scanning dependencies of target main
[100%] Building CXX object CMakeFiles/main.dir/main.cpp.o
Linking CXX executable main
src/libpagmo.a(jde.cpp.o): In function `void boost::serialization::throw_exception<boost::archive::archive_exception>(boost::archive::archive_exception const&)':
jde.cpp:(.text._ZN5boost13serialization15throw_exceptionINS_7archive17archive_exceptionEEEvRKT_[_ZN5boost13serialization15throw_exceptionINS_7archive17archive_exceptionEEEvRKT_]+0x1d): undefined reference to `boost::archive::archive_exception::archive_exception(boost::archive::archive_exception const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [main] Error 1
make[1]: *** [CMakeFiles/main.dir/all] Error 2
make: *** [all] Error 2

In conclusion, the instructions work when you are not building main, but fail if you also want to build main.

Thanks

Akshay

[darioizzo]

Did you also invert the order of the linked libraries when building main? It seems not. And if you build PyGMO you need to do it there too ....

[akshayc11]

This is what I did for the CMakeList.txt for pagmo:

speech@gpu04:/data-local/akshayc/Workspace/Software/pagmo$ git diff CMakeLists.txt
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7a0a5c9..201047e 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -394,7 +394,7 @@ MESSAGE(STATUS "Shared linker flags: " "${CMAKE_SHARED_LINKER_FLAGS}")
 # Link main to pagmo_static library.
 IF(BUILD_MAIN)
        ADD_EXECUTABLE(main main.cpp)
-               TARGET_LINK_LIBRARIES(main ${MANDATORY_LIBRARIES} pagmo_static)
+               TARGET_LINK_LIBRARIES(main pagmo_static ${MANDATORY_LIBRARIES} ${MANDATORY_BOOST_LIBS})
 ENDIF(BUILD_MAIN)

For The CMakeLists.txt within PyGMO:

speech@gpu04:/data-local/akshayc/Workspace/Software/pagmo$ git diff CMakeLists.txt
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7a0a5c9..201047e 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -394,7 +394,7 @@ MESSAGE(STATUS "Shared linker flags: " "${CMAKE_SHARED_LINKER_FLAGS}")
 # Link main to pagmo_static library.
 IF(BUILD_MAIN)
        ADD_EXECUTABLE(main main.cpp)
-               TARGET_LINK_LIBRARIES(main ${MANDATORY_LIBRARIES} pagmo_static)
+               TARGET_LINK_LIBRARIES(main pagmo_static ${MANDATORY_LIBRARIES} ${MANDATORY_BOOST_LIBS})
 ENDIF(BUILD_MAIN)

 IF(ENABLE_TESTS)
speech@gpu04:/data-local/akshayc/Workspace/Software/pagmo$ emacs CMakeLists.txt
speech@gpu04:/data-local/akshayc/Workspace/Software/pagmo$ git diff PyGMO/CMakeLists.txt
diff --git a/PyGMO/CMakeLists.txt b/PyGMO/CMakeLists.txt
index 76ce754..206ab20 100644
--- a/PyGMO/CMakeLists.txt
+++ b/PyGMO/CMakeLists.txt
@@ -1,8 +1,8 @@
-SET(MANDATORY_LIBRARIES pagmo ${MANDATORY_LIBRARIES})
+SET(MANDATORY_LIBRARIES ${MANDATORY_LIBRARIES} pagmo )
 IF(PYTHON_VERSION_MAJOR LESS 3)
-       SET(MANDATORY_LIBRARIES ${MANDATORY_LIBRARIES} ${Boost_PYTHON_LIBRARY})
+       SET(MANDATORY_LIBRARIES ${Boost_PYTHON_LIBRARY} ${MANDATORY_LIBRARIES} )
 ELSE(PYTHON_VERSION_MAJOR LESS 3)
-       SET(MANDATORY_LIBRARIES ${MANDATORY_LIBRARIES} ${Boost_PYTHON3_LIBRARY})
+       SET(MANDATORY_LIBRARIES ${Boost_PYTHON3_LIBRARY} ${MANDATORY_LIBRARIES} )
 ENDIF(PYTHON_VERSION_MAJOR LESS 3)

 SET(PYGMO_INSTALL_PATH ${PYTHON_MODULES_DIR}/PyGMO)
@@ -23,7 +23,7 @@ MACRO(PYTHON_CPPMODULE arg1)
        ELSEIF(SOEXTENSION)
                SET_TARGET_PROPERTIES(${arg1} PROPERTIES SUFFIX ".so")
        ENDIF(PYDEXTENSION)
-       TARGET_LINK_LIBRARIES(${arg1} ${ARGN} ${MANDATORY_LIBRARIES} ${PYTHON_LIBRARY})
+       TARGET_LINK_LIBRARIES(${arg1} ${ARGN} ${PYTHON_LIBRARY} ${MANDATORY_LIBRARIES})
        STRING(REGEX REPLACE "^_" "" DIR_ARG ${arg1})
        INSTALL(TARGETS ${arg1}
                RUNTIME DESTINATION ${PYGMO_INSTALL_PATH}/${DIR_ARG}
@@ -39,7 +39,7 @@ MACRO(PYTHON_CPPMODULE_ADDITION arg1 arg2)
        ELSEIF(SOEXTENSION)
                SET_TARGET_PROPERTIES(${arg1} PROPERTIES SUFFIX ".so")
        ENDIF(PYDEXTENSION)
-       TARGET_LINK_LIBRARIES(${arg1} ${ARGN} ${MANDATORY_LIBRARIES} ${PYTHON_LIBRARY})
+       TARGET_LINK_LIBRARIES(${arg1} ${ARGN} ${PYTHON_LIBRARY} ${MANDATORY_LIBRARIES})
        STRING(REGEX REPLACE "^_" "" DIR_ARG ${arg1})
        INSTALL(TARGETS ${arg1}
                RUNTIME DESTINATION ${PYGMO_INSTALL_PATH}/${arg2}

I still get the same error:

import PyGMO
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/PyGMO/__init__.py", line 57, in <module>
    from PyGMO import core, algorithm, migration, problem, topology, test, util
  File "/usr/lib/python2.7/dist-packages/PyGMO/core/__init__.py", line 2, in <module>
    from PyGMO.core._core import *
ImportError: /usr/local/lib/libpagmo.so: undefined symbol: _ZN5boost7archive17archive_exceptionC1ERKS1_
>>>

Is there anything I am missing?

[darioizzo]

Try to build only main.cpp and swap the order of linking. Does this work?

[akshayc11]

Hi Dario,
This is what I did:

git diff CMakeLists.txt 
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7a0a5c9..97d7010 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -394,7 +394,7 @@ MESSAGE(STATUS "Shared linker flags: " "${CMAKE_SHARED_LINKER_FLAGS}")
 # Link main to pagmo_static library.
 IF(BUILD_MAIN)
        ADD_EXECUTABLE(main main.cpp)
-               TARGET_LINK_LIBRARIES(main ${MANDATORY_LIBRARIES} pagmo_static)
+               TARGET_LINK_LIBRARIES(main pagmo_static ${MANDATORY_LIBRARIES})
 ENDIF(BUILD_MAIN)

 IF(ENABLE_TESTS)

Then I ran the following:

mkdir build
cd build
cmake ..
make

This results in:

... #(not printing the success messages during build)
[100%] Built target pagmo_static
Scanning dependencies of target main
[100%] Building CXX object CMakeFiles/main.dir/main.cpp.o
Linking CXX executable main
src/libpagmo.a(jde.cpp.o): In function `void boost::serialization::throw_exception<boost::archive::archive_exception>(boost::archive::archive_exception const&)':
jde.cpp:(.text._ZN5boost13serialization15throw_exceptionINS_7archive17archive_exceptionEEEvRKT_[_ZN5boost13serialization15throw_exceptionINS_7archive17archive_exceptionEEEvRKT_]+0x1d): undefined reference to `boost::archive::archive_exception::archive_exception(boost::archive::archive_exception const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [main] Error 1
make[1]: *** [CMakeFiles/main.dir/all] Error 2
make: *** [all] Error 2

This is the only change and corresponding error I get while compiling pagmo_static

I do not know what else to change, or why this is happening. Do I need to change anything else?

I am running this on a ubuntu 14.04 machine with boost 1.61 and gcc 4.8.4.
Boost has been installed with no problems.

[darioizzo]

can you run

make verbose=5

or

make -n

and post the output

[akshayc11]

Hi Dario,

Here is the pastebin for both of those commands:
http://pastebin.com/0sC1jriH

the make verbose=5 seemed to do nothing different.

[akshayc11]

Maybe this may be more useful:
Here are the contents of the link.txt file in build/CMakeFiles/main.dir/

usr/bin/c++ -Wall -Wno-deprecated -Wextra -Wnoexcept -Wdisabled-optimization -ffast-math -std=c++11 -pthread -DBOOST_THREAD_USE_DLL -DBOOST_SERIALIZATION_DYN_LINK=1 -O3 -DNDEBUG CMakeFiles/main.dir/main.cpp.o -o main -rdynamic
src/libpagmo.a /usr/local/lib/libboost_system-mt.so /usr/local/lib/libboost_thread-mt.so /usr/local/lib/libboost_serialization-mt.so -lpthread -Wl,-rpath,/usr/local/lib

[darioizzo]

I am not sure what is happening in your system. Clearly somehow the serialization library is not linked in correctly when building main.cpp. I am trying to reproduce your problem but it seems I cannot, so I do not really know how to help much.

Did you try to do everything from scratch? Like clean the build directory, run ccmake ../ select only build main and make?

Another thing to try is to manually run the g++ command swapping the library order at the end ....

[akshayc11]

Hi Dario, yes, I did everything from scratch. But it still did not work. In any case, I will continue looking into this myself, and see if I can fix it. If I do fix it, I will let you know how I did so.
Thanks a lot for all the assistance.

Sincerely,
Akshay

[darioizzo]

fyi this is the output of our CI system https://travis-ci.org/esa/pagmo pagmo build also in our osx machine and archlinux.

Let me know

[ChrisAndre]

I haven't been following the project for a long time (disclaimer), but I might have something worthwhile:

I've previously had trouble with building PaGMO and friends with the Boost libraries installed with boost.org's source install scripts. That trouble, as I remember, manifests as linking difficulties with Boost (after flipping the link order in PaGMO+ as required). Unfortunately, I do not know why that happens. CMake appears happy; I'm not very familiar with make and the linking after that.

I've found that the sure-fire way to install PaGMO+ on Ubuntu 14.04 without issues is to rely on apt-get to install Boost, instead of the manual source installer from boost.org. If I recall, the catch-all package is libboost-all-dev. The packaged Boost might be an older version, though.

I don't know why the package installation tends to work while the source installation fails. But hopefully that can at least solve the build issue.

[akshayc11]

I got it working with OSX in a different machine. As long as I have one instance, I am fine. I will try the boost installation again if I really need it.

Thanks for all the tips.

Sincerely,
Akshay

[gmazzamuto]

Hi all,
I was having exactly the same problem on Ubuntu 16.04 and I solved it by putting the static library first. I just saw the previously closed issues on this same topic (e.g. #68, #69). As far as I understand, inverting the linking order didn't make it to the master branch because it breaks linking in distributions such as fedora or archlinux. However, I was able to successfully build pagmo with the modified order on the latest docker images of ubuntu, archlinux and fedora. If you think these changes can be useful, I will submit a pull request. Otherwise maybe it could be useful to mention this build issue explicitly in the installation guide.
Thanks,
Giacomo

[darioizzo]

You can issue a pull request. If it builds on our CI system I will also give it a try

[RockyRocks]

Can we not do away with this boost dependency?