Building mbgl-native with g++ against binary libs built with clang++

[springmeyer]

On Linux some users will prefer clang++ and others g++. So we want to support both compilers, as long as they are recent enough to support c++14.

Note: On OS X life is easy because only clang++ is used commonly. And despite there being both libc++ and libstdc++ present on OS X, Apple has worked to make them binary compatible (so applications can link one and then link a dependent library that links another without problems).

Mixing these two compilers is generally fine in my experience because clang++ aims at g++ compatibility. But in the end they are different compilers. So subtly different compiler flags may behave different and linking binaries from each is not 100% since even if things work today, they may not tomorrow as these compilers iterate quickly.

Let's use this ticket as a master ticket to flesh out where mixing compilers breaks down. Let's restrict scope to mbgl code and the specific deps we require. And to c++14 and just recent clang++ and g++ versions. And let's ignore -flto which is notoriously non compatible/portable. Let's get a good working sense of these questions:

• In what cases can binaries with -std=c++11 and -std=c++14 be mixed or not mixed.
• In what cases can binaries made with clang++ and g++ be mixed or not mixed and for what versions?
• How does -fvisibility=hidden and -fvisibility-inlines-hidden pay into this complexity?

Background:

• Fix npm install from source #2237 (comment)
• Design a system to provide binaries of C++ libraries built for different ABIs mason#111

[mikemorris]

Linux/gcc linker errors appear to be fixed when building with g++-5 https://travis-ci.org/mapbox/mapbox-gl-native/builds/83786608

[jfirebaugh]

Here is a linux build with the link error:

./Release/obj.target/libmbgl-core.a(annotation_manager.o): In function `mbgl::AnnotationManager::addShapeAnnotation(mbgl::ShapeAnnotation const&, unsigned char)':
/home/travis/build/mapbox/mapbox-gl-native/build/linux-x86_64/../../src/mbgl/annotation/annotation_manager.cpp:133: undefined reference to `mapbox::util::geojsonvt::Convert::create(std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > >, mapbox::util::geojsonvt::ProjectedFeatureType, mapbox::util::variant<mapbox::util::geojsonvt::ProjectedPoint, mapbox::util::geojsonvt::ProjectedGeometryContainer>)'

The command that compiles geojson_convert.cpp is:

clang++-3.5 '-DNDEBUG' -I../include -isystem /home/travis/build/mapbox/mason/mason_packages/.build/geojson-vt-cpp-2.1.6.1/mason_packages/headers/variant/1.0/include -I/home/travis/build/mapbox/mason/mason_packages/.build/geojson-vt-cpp-2.1.6.1/mason_packages/headers/variant/1.0/include -I/home/travis/build/mapbox/mason/mason_packages/.build/geojson-vt-cpp-2.1.6.1/mason_packages/headers/rapidjson/1.0.2/include -g -O3 -std=c++1y -MMD -MF ./Release/.deps/Release/obj.target/geojsonvt/src/geojsonvt_convert.o.d.raw -fPIC -fPIC -fPIC -std=c++11 -c -o Release/obj.target/geojsonvt/src/geojsonvt_convert.o ../src/geojsonvt_convert.cpp

The command that compiles annotation_manager.cpp (which includes geojson_convert.hpp) is:

g++-4.9 '-DNDEBUG' -I../../include -I../../src -I./Release/obj/gen/include -I./Release/obj/gen/include -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/linux-x86_64/libuv/0.10.28/include -isystem /home/travis/build/mapbox/mapbox-gl-native/mason_packages/headers/variant/1.0/include -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/headers/variant/1.0/include -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/headers/rapidjson/1.0.2/include -fPIC -std=c++14 -Werror -Wall -Wextra -Wshadow -Wno-variadic-macros -Wno-error=unused-parameter -Wno-c++1y-extensions -frtti -fexceptions -Wno-unknown-pragmas -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/linux-x86_64/libuv/0.10.28/include -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/headers/boost/1.57.0/include -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/linux-x86_64/geojsonvt/2.1.6.1/include -isystem /home/travis/build/mapbox/mapbox-gl-native/mason_packages/headers/variant/1.0/include -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/headers/variant/1.0/include -I/home/travis/build/mapbox/mapbox-gl-native/mason_packages/headers/rapidjson/1.0.2/include -fPIC -g -O3 -MMD -MF ./Release/.deps/Release/obj.target/core/src/mbgl/annotation/annotation_manager.o.d.raw -c -o Release/obj.target/core/src/mbgl/annotation/annotation_manager.o ../../src/mbgl/annotation/annotation_manager.cpp

Dumping symbols from libgeojsonvt.a with nm -C reveals that there is indeed no mapbox::util::geojsonvt::Convert::create symbol in geojsonvt_convert.o. The symbol that is intended to match is probably this one, which apparently can't be demangled:

0000000000004410 T _ZN6mapbox4util9geojsonvt7Convert6createESt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEENS1_20ProjectedFeatureTypeENS0_7variantIJNS1_14ProjectedPointENS1_26ProjectedGeometryContainerEEEE

[jfirebaugh]

http://demangler.com/ is able to demangle that symbol, and it matches the name of the symbol that GCC thinks is undefined.

My hunch is this is simply a compiler bug, in that they don't agree what the mangled name of the symbol should be, though I'm not sure whether it's clang-3.5's or GCC's fault.

I'm going to build mbgl again with LDFLAGS=--no-demangle to see what mangled name GCC expects.

[jfirebaugh]

Had to use COLLECT_NO_DEMANGLE=1 because gcc didn't understand --no-demangle, but the symbol that GCC is looking for is... drumroll please:

_ZN6mapbox4util9geojsonvt7Convert6createESt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEENS1_20ProjectedFeatureTypeENS0_7variantIINS1_14ProjectedPointENS1_26ProjectedGeometryContainerEEEE

Compare that to the symbol that clang-3.5 generated:

_ZN6mapbox4util9geojsonvt7Convert6createESt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEENS1_20ProjectedFeatureTypeENS0_7variantIJNS1_14ProjectedPointENS1_26ProjectedGeometryContainerEEEE

I've highlighted the one character difference.

[jfirebaugh]

The I vs J is related to how template argument packs are mangled. Knowing that told me what to search for, and then I found https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58591 / https://gcc.gnu.org/bugzilla/show_bug.cgi?id=51322.

Conclusion: this is a GCC ABI bug, which can be fixed by compiling in GCC with -fabi-version=6 or -fabi-version=0.

Documentation for -fabi-version states:

Version 6, which first appeared in G++ 4.7, corrects the promotion behavior of C++11 scoped enums and the mangling of template argument packs, const/static_cast, prefix ++ and –, and a class scope function used as a template argument.

Note that clang does not have an equivalent of -fabi-version. More fascinating reading:

• https://llvm.org/bugs/show_bug.cgi?id=24685
• http://clang-developers.42468.n3.nabble.com/GXX-ABI-VERSION-td4046114.html

"Clang doesn't have a notion of any C++ ABI version. We make bugfixes all the time and mostly just pray it only affects non-extern template instantiations."

Fun times.

[jfirebaugh]

Confirmed that -fabi-version=0 fixes the link error on linux gcc. However it causes clang builds to fail (clang: error: unknown argument: '-fabi-version=0').

Since -fabi-version=0 is the default for GCC 5, perhaps upgrading is the best course of action: #2553.