Missing symbols in libLLVM.so when built with clang-4.0

[tstellar]

Bugzilla Link	33161
Version	trunk
OS	Linux
Attachments	Test Program, Makefile for testcase
CC	@chandlerc,@DougGregor,@zygoloid,@jwakely

Extended Description

There are some symbols missing from libLLVM.so when built with clang-4.0. I can reproduce this when building libLLVM.so from trunk (r303802) and libLLVM.so from the llvm 4.0 source. I cannot reproduce this when libLLVM.so is built with gcc-4.8.5.

I think the issue has something to do with the use of 'extern template class AnalysisManager<Loop, LoopStandardAnalysisResults &>;' in LoopAnalysisManager.h, but I'm not sure if it is a bug in the LLVM source code that just happens to work when compiled by gcc, or if it is a miscompile by clang.

I was looking at the difference between the clang built and llvm built shared objects and I see:

$ nm --demangle libLLVM-4.0.so.clang4.0 \
| grep 'llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)'

00000000013af5f0 W llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)
00000000013af5f0 W llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)

$ nm --demangle check-for-broken-symbol.sh libLLVM-4.0.so.gcc4.8.5
| grep 'llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)'

00000000013f12d0 W llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)
00000000013f12d0 W llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)
00000000013f12d0 t llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)
00000000013f12d0 t llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)

Attached is a simple program to demonstrate the issue. Make sure to set LLVM_LIB_PATH and LLVM_INCLUDE_PATH in the Makefile to the correct paths for your system, and make sure that libLLVM.so has been built by clang.

To reproduce:

$ make gcc-main
gcc -D__STDC_CONSTANT_MACROS -std=c++11 -o gcc-main.o -c main.cpp -I /home/tstellar/llvm-project/llvm-install/include/
gcc -o gcc-main -L /home/tstellar/llvm-project/llvm-install/lib -lLLVM-4.0 -lstdc++ gcc-main.o
gcc-main.o: In function main': main.cpp:(.text+0x1c): undefined reference to llvm::AnalysisManager<llvm::Loop, llvm::LoopStandardAnalysisResults&>::AnalysisManager(bool)'

I also noticed that if I build main.cpp with clang, then the program links correctly:

$ make clang-main
clang -D__STDC_CONSTANT_MACROS -std=c++11 -o clang-main.o -c main.cpp -I /usr/include/
gcc -o clang-main -L /usr/lib -lLLVM -lstdc++ clang-main.o

[chandlerc]

I dunno what is going on here. There is definitely a bool accepting constructor of this class template and that class template seems to have an explicit instantiation. Maybe Richard can explain why the code is buggy if Clang is correct here?

[zygoloid]

It looks like Clang is emitting a definition of the symbol and it's surviving into the .so file, based on the nm output in comment#0. (The 't' entries should not be relevant, since they're not external symbols.) What do you see if you use 'nm -D' instead, to query the dynamic symbol table?

The fact that main.cpp works when compiled by Clang but not when compiled by GCC makes me wonder if the compilers are disagreeing on the mangling of this symbol somehow. Please also remove the --demangle from your nm invocations to check.

I'd also be interested in the nm output on the two ...-main.o files; do they have undefined references to the same symbol?

[tstellar]

It looks like the symbols are mangled differently by the two compilers:

clang:

$ nm clang-main.o | grep 7LoopStandardAnalysisResultsEEEC1Eb
U _ZN4llvm15AnalysisManagerINS_4LoopEJRNS_27LoopStandardAnalysisResultsEEEC1Eb

$ nm -D ../clang/libLLVM-4.0.so.clang4.0 | grep 7LoopStandardAnalysisResultsEEEC1Eb
00000000013af5f0 W _ZN4llvm15AnalysisManagerINS_4LoopEJRNS_27LoopStandardAnalysisResultsEEEC1Eb

gcc:

$ nm gcc-main.o | grep 7LoopStandardAnalysisResultsEEEC1Eb
U _ZN4llvm15AnalysisManagerINS_4LoopEIRNS_27LoopStandardAnalysisResultsEEEC1Eb

$ nm -D ../gcc/libLLVM-4.0.so.gcc4.8.5 | grep 7LoopStandardAnalysisResultsEEEC1Eb
00000000013f12d0 W _ZN4llvm15AnalysisManagerINS_4LoopEIRNS_27LoopStandardAnalysisResultsEEEC1Eb

[jwakely]

It's a variadic template, so J is correct:

template <typename IRUnitT, typename... ExtraArgTs> class AnalysisManager;

It looks like all versions of GCC up to 6.1 emit some symbols with I in place of J in the mangled names.

This might be https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59457

[jwakely]

This was https://gcc.gnu.org/#51322 and GCC's mangling was fixed for -fabi-version=6 in version 4.7

With -fabi-version=6 GCC emits the right symbol, otherwise it looks to me like it emits two symbols, with the correctly mangled one aliased to the incorrectly mangled one. Maybe the alias is being lost.

[zygoloid]

Looks like GCC emits an alias with the definition of the symbol, but not with the use of the symbol[1]. So if the definition is compiled by a compiler that produces the correct mangling, as in this case, linking will fail. :(

[1] This is hard, since ELF doesn't support aliases to unresolved symbols. I suppose you could try something crazy like emitting an ifunc that tries to resolve both symbols on startup.

If we want to be link-compatible with this GCC mangling bug, it looks like the only path forward is to implement GCC's bug and generate an alias ourselves too. Yuck. As a less ugly alternative, perhaps we could include the -fabi-version=6 workaround in the release notes.

[tstellar]

Here is a documentation patch describing this bug: https://reviews.llvm.org/D34212

[fhahn]

Bug in GCC, closing.