Unable to compile with LTO on Ubuntu 14.04 (with ubuntu-toolchain-r ppa)

[daniel-j-h]

Splitting off from #3269 (comment) - this is a separate issue. @karenzshea reported seeing:

[ 56%] Linking CXX executable osrm-datastore
/tmp/ccfdAtPm.ltrans20.ltrans.o:<artificial>:function main: error: undefined reference to 'osrm::storage::StorageConfig::StorageConfig(boost::filesystem::path const&)'
/tmp/ccfdAtPm.ltrans20.ltrans.o:<artificial>:function main: error: undefined reference to 'osrm::storage::StorageConfig::IsValid() const'
/tmp/ccfdAtPm.ltrans20.ltrans.o:<artificial>:function main: error: undefined reference to 'osrm::storage::Storage::Storage(osrm::storage::StorageConfig)'
/tmp/ccfdAtPm.ltrans20.ltrans.o:<artificial>:function main: error: undefined reference to 'osrm::storage::Storage::Run(int)'
collect2: error: ld returned 1 exit status
make[2]: *** [osrm-datastore] Error 1
make[1]: *** [CMakeFiles/osrm-datastore.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

-- The C compiler identification is GNU 4.9.4
-- The CXX compiler identification is GNU 4.9.4
-- Check for working C compiler: /usr/bin/gcc-4.9
-- Check for working C compiler: /usr/bin/gcc-4.9 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-4.9
-- Check for working CXX compiler: /usr/bin/g++-4.9 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building on a 64 bit system
-- Using GNU gold as linker.
-- Disabling gc-sections on gold binutils < 2.26, see: https://sourceware.org/bugzilla/show_bug.cgi?id=17639
-- No build type specified, defaulting to Release
-- Configuring OSRM in release mode
-- Configuring release mode optimizations
-- Performing Test LTO_AVAILABLE
-- Performing Test LTO_AVAILABLE - Success
-- Performing Test LTO_WORKS
-- Performing Test LTO_WORKS - Success
-- LTO working
CMake Warning at CMakeLists.txt:231 (message):
  GCC specific binutils not found.

-- Performing Test HAS_COLOR_FLAG
-- Performing Test HAS_COLOR_FLAG - Success
-- Setting linker optimizations
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Boost version: 1.54.0
-- Found the following Boost libraries:
--   date_time
--   chrono
--   filesystem
--   iostreams
--   program_options
--   regex
--   system
--   thread
--   unit_test_framework
--   atomic
-- Found Intel TBB
-- TBB interface version: 7000
-- Found Lua: /usr/lib/x86_64-linux-gnu/liblua5.2.so;/usr/lib/x86_64-linux-gnu/libm.so (found suitable exact version "5.2.3") 
-- Using Lua 5.2.3
-- Found Luabind: /usr/lib/libluabind.so  
-- Found Luabind: /usr/lib/libluabind.so
-- Found EXPAT: /usr/lib/x86_64-linux-gnu/libexpat.so (found version "2.1.0") 
-- Looking for STXXL...
-- Found STXXL: /usr/lib/libstxxl.so  
-- Found STXXL: /usr/lib/libstxxl.so
-- Found BZip2: /usr/lib/x86_64-linux-gnu/libbz2.so (found version "1.0.6") 
-- Looking for BZ2_bzCompressInit
-- Looking for BZ2_bzCompressInit - found
-- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE) 
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8") 
-- Looking for protozero
-- Looking for protozero - found
-- Found Osmium: /mnt/osrm-backend/third_party/libosmium/include  
-- OpenMP support found. Linking just in case for stxxl
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/osrm-backend/build

[daniel-j-h]

CMake Warning at CMakeLists.txt:231 (message):
  GCC specific binutils not found.

Here's what I think is going on: with LTO object files include compiler-specific intermediate representation. Your linker then has to understand this internal representation to run the optimization passes on and emit machine code in the codegen phase.

That's the reason GCC ships gcc-ar, gcc-nm and gcc-ranlib since GCC 4.9 which we have to use when enabling link-time optimization. We try to detect these binaries here:

osrm-backend/CMakeLists.txt

Lines 225 to 227 in f1384f5

	# Since gcc 4.9 the LTO format is non-standart ('slim'), so we need to use the build-in tools
	if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND
	NOT "${CMAKE_CXX_COMPILER_VERSION}" VERSION_LESS "4.9.0" AND NOT MINGW)

(and probably should fail if LTO is on and we can not detect them)

@karenzshea can you investigate why you have a gcc 4.9.4 but not these gcc-specific binaries? From what I can tell Trusty has gcc 4.8 (ref.) for which we disable LTO altogether.

You're probably using the ubuntu-toolchaiun-r ppa? If you scroll down to gcc-4.9-4.9.4-2ubuntu1~14.04.1 (for Trusty in the column to the right) and expand the item you will find gcc-4.9_4.9.4-2ubuntu1~14.04.1_amd64.deb which comes with the gcc specific binaries. This is what you get when you apt-get install gcc.

So my guess for now is you already have the gcc specific binaries but they are version-suffixed (think gcc-ar-4.9) and we don't pick them up. And later compiler versions no longer append the version (e.g. for gcc5, gcc6, etc.).

[TheMarex]

That's the reason GCC ships gcc-ar, gcc-nm and gcc-ranlib since GCC 4.9 which we have to use when enabling link-time optimization.

These are actually only needed for older versions of binutils as per https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ On my system using binutil 2.27 it does not matter which one is used.

That is the primary reason we are not failing by default of they are not detected. I think we should just disable LTO by default for everything below gcc 5 and clang 3.8

[daniel-j-h]

I just came across #2006 - which describes exactly the problem here. This is not related to specific gcc version. Instead it's related to some distributions shipping version-suffixed tools such as gcc-ar-4.9.

[daniel-j-h]

Closing as Ubuntu 14 + ubuntu-toolchain-r/test issue. Here's a workaround:

export AR=gcc-ar-4.9
export NM=gcc-nm-4.9
export RANLIB=gcc-ranlib-4.9

Encoding this in the CMakeLists now.

[springmeyer]

That's the reason GCC ships gcc-ar, gcc-nm and gcc-ranlib since GCC 4.9 which we have to use when enabling link-time optimization.

These are actually only needed for older versions of binutils as per https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ On my system using binutil 2.27 it does not matter which one is used.

https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ is great to read/see @TheMarex. I'll keep a closer eye on this in the future to see what actually requires llvm-ar and llvm-ranlib in the clang++ case, but at least what I've seen is that binutils 2.27's ar does not work to create libosrm.a (don't have the error around anymore). Also btw for llvm llvm-ranlib is a symlink to llvm-ar and llvm-nm does not appear to be used or needed.

[rhalbersma]

Ubuntu 14.04 has a binutils-2.26 package that can be installed alongside the distro's native binutils (which is version 2.24). Then with export PATH=/usr/lib/binutils-2.26/bin:${PATH}, the 2.26 version will be found before the 2.24 version. Works for me on Travis CI using gcc-7.