Clang and Boost in Ubuntu 16.04

[costashatz]

When compiling DART 6.3, 6.4 or the master under Ubunut 16.04 with Clang, I am getting some weird linker errors related to Boost. The library builds nicely but when I link to it everything fails. With gcc, it works like a charm. Digging up a bit more, I found this bug. I tried with Clang 3.8 and 4.0 and I get the same thing. Just making this issue if anyone gets the same error. The solution is to use gcc or update gcc to newer version in order to use clang.

[mxgrey]

Were you getting this for boost::regex rather than boost::program_options? If so, I saw it as well.

I think the solution might be to seek out a more robust way to decide between using boost::regex vs std::regex. The decision is currently being made by checking a glibc macro __GLIBCXX__, which errs towards always using boost::regex.

I wonder if it may be possible to make the decision in a more robust way inside of cmake, and then pass along a compile definition to decide whether std or boost is used.

[costashatz]

Were you getting this for boost::regex rather than boost::program_options?

Yes it was boost::regex..

I wonder if it may be possible to make the decision in a more robust way inside of cmake, and then pass along a compile definition to decide whether std or boost is used.

We can always do the check in CMake and make a define of our own to ensure that everything is well defined..

[mxgrey]

Yeah, I'm just not clear on whether cmake has the necessary information to decide whether we need to use boost::regex or std::regex. It's apparently not a very trivial thing, according to the comment in the code where boost::regex is being used.

@mkoval might be able to shed some light on the issue if he remembers the circumstances surrounding the old issue. From what I can gather, compiling with clang makes it difficult to identify the version of libstdc++ that we're linking against. We need it to be version 6.0.20 or higher in order to safely use std::regex, because earlier versions were providing a buggy version of std::regex.

If cmake had a way for us to recognize what version of libstdc++ we link against, then we could handle this pretty elegantly during configure time. Off the top of my head, I don't know a cmake mechanism for checking that.

[costashatz]

Since the problem only appears with Clang and linux distributions, one quick fix would be to undefine __GLIBCXX__ when we detect Clang under linux distros (or create specific defines for this regex thing; it might not be a good idea to undefine __GLIBCXX__). This is of course assuming that Clang implements std::regex correctly (which I am not sure; there seems to be a small ambiguity in Clang version <= 3.8; see here). I tested this trick with Clang 3.8 and 4.0 and it works like a charm..

Another more robust check would be to do some CMake magic to detect the actual version of libstdc++ (see here) and then create some defines depending on the libstdc++ version, compiler and distro. Since this problem appears only with Clang and linux distros, it should not be too hard to do..

I am not an expert of these things, so correct me if I am wrong anywhere..!

[psigen]

Well, one option would be to use CMake's compile test functionality to exploit this particular condition:

IIRC std::regex_search always returns false in the buggy version.

https://stackoverflow.com/questions/12530406/is-gcc-4-8-or-earlier-buggy-about-regular-expressions/12665408#comment56934397_12665408

We could use CheckCXXSourceRuns with source code that throws an exception for the above case and set a flag based on that?
https://cmake.org/cmake/help/v3.0/module/CheckCXXSourceRuns.html?highlight=checkcxx#module:CheckCXXSourceRuns

[mxgrey]

For the record, I really like @psigen 's solution for this. It sounds like the most robust and straightforward way to deal with this bizarre and tricky issue.

[mosra]

Hi! I encountered a similar case and here's what I discovered -- hopefully it'll be of some use for you (if not, then sorry for the noise).

In Magnum I have an integration for DART (contributed by @costashatz) and when upgrading to Travis CI 16.04 (from 14.04) I started getting a similar linker issue -- however with GCC 4.8 and not Clang. I have a bit special case, since I need to maintain GCC 4.8 compatibility there (for people running Magnum server-side on CentOS which is still on 4.8) yet OTOH I need 16.04 for newer packages like Eigen.

Here is the linker error in full:

FAILED: : && /usr/bin/g++-4.8 --coverage -g src/Magnum/DartIntegration/Test/CMakeFiles/DartIntegrationConvertShapeNodeTest.dir/ConvertShapeNodeTest.cpp.o -o src/Magnum/DartIntegration/Test/DartIntegrationConvertShapeNodeTest -rdynamic src/Magnum/DartIntegration/libMagnumDartIntegration-d.so.2.2 /home/travis/deps-dart/lib/libdart-utils-urdfd.so.6.3.0 /home/travis/deps/lib/libCorradeTestSuite-d.so /home/travis/deps/lib/libMagnumSceneGraph-d.so /home/travis/deps/lib/libMagnumPrimitives-d.so /home/travis/deps/lib/libMagnumMeshTools-d.so /home/travis/deps/lib/libMagnumGL-d.so -lGL /home/travis/deps/lib/libMagnumTrade-d.so /home/travis/deps/lib/libMagnum-d.so /home/travis/deps/lib/libCorradePluginManager-d.so -ldl /home/travis/deps-dart/lib/libdart-utilsd.so.6.3.0 /home/travis/deps-dart/lib/libdartd.so.6.3.0 /home/travis/deps-dart/lib/libccd.so /home/travis/deps-dart/lib/libfcl.so -lassimp -lboost_regex -lboost_system /home/travis/deps-dart/lib/libdart-external-odelcpsolverd.so.6.3.0 -ltinyxml -ltinyxml2 -lurdfdom_sensor -lurdfdom_model_state -lurdfdom_model -lurdfdom_world -lconsole_bridge /home/travis/deps/lib/libCorradeUtility-d.so -Wl,-rpath,/home/travis/build/mosra/magnum-integration/build/src/Magnum/DartIntegration:/home/travis/deps-dart/lib:/home/travis/deps/lib && :
/home/travis/deps-dart/lib/libdartd.so.6.3.0: undefined reference to 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits > >::find()'
/home/travis/deps-dart/lib/libdartd.so.6.3.0: undefined reference to 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits > >::match()'
/home/travis/deps-dart/lib/libdart-utils-urdfd.so.6.3.0: undefined reference to 'urdf::parseURDF(std::string const&)'
/home/travis/deps-dart/lib/libdartd.so.6.3.0: undefined reference to 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits > >::construct_init(boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits > > const&, boost::regex_constants::_match_flags)'
collect2: error: ld returned 1 exit status

Note the (highlighted) linker error to urdf::parseURDF -- it took me a while to spot, but then I realized both urdfdom and boost_regex are coming from system packages (liburdfdom-dev and libboost-regex-dev), while I'm compiling the rest (libccd, flann, dart) myself (can't add a PPA because the containers are sudo-less etc. etc.). Digging further, I remembered that there was an ABI change affecting std::string going from GCC 4.8 to GCC 5 and, as strange as it may seem, in this case it affected only these few string-related symbols -- it should be looking for std::__cxx11::basic_string instead of std::string. Solutions I could think of (for Magnum itself) include:

• Downgrade the distro or upgrade the compiler.
• Compile both boost_regex and urdfdom manually using GCC 4.8 as well.
• Convince GCC 4.8 to use the GCC 5 ABI. That's a very dangerous hack.

Neither of these are feasible in my case, so I'll be disabling DART builds on Linux and enabling them on macOS instead (I hope brew install libdart "just works", hehe); and revisiting this at some unspecified point in the future when I have a reason to bump minimal compiler versions again.

Regarding Clang -- on Linux, Clang is using libstdc++ by default (you have to explicitly tell it to use libc++, which is the default on Apple platforms), meaning all std::regex bugs in old GCCs affect it as well -- so enabling boost::regex just for GCC won't fix anything. The linker error is, as far as I can tell, a result of the same GCC 5 ABI change. The relevant bug report seems to suggest that Clang 3.9 and up is aware of the GCC 5 ABI change, but other sources say that it's still broken in 4.0, so ¯\_(ツ)_/¯.

Hope this helps :)

EDIT: btw. if you say that you need at least libstdc++ 6 for std::regex -- since GCC 7 there's a new _GLIBCXX_RELEASE macro (which you can get from #include <ciso646> or any other STL header, this one is the smallest), so you could check this way. It's defined to major GCC version, so 7, 8, 9... It will skip GCC 6, but I think that's futureproof-enough.

[jslee02]

Closing as Boost dependency is completely removed by DART 6.13!