relocation R_X86_64_PC32 against symbol `_ZTVN4llvm11raw_ostreamE@@LLVM_7' can not be used

[danstender]

Hi,
trying to build llvmlite 0.27.1rc1 this build break came up:

++  -shared -I/usr/lib/llvm-7/include -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -g -flto assembly.cpp bitcode.cpp core.cpp initfini.cpp module.cpp value.cpp executionengine.cpp transforms.cpp passmanagers.cpp targets.cpp dylib.cpp linker.cpp object_file.cpp -o libllvmlite.so -Wl,-z,relro -L/usr/lib/llvm-7/lib -flto -Wl,--exclude-libs=ALL -lLLVM-7
transforms.cpp:10:32: warning: 'unwrap' violates the C++ One Definition Rule  [-Wodr]
     inline PassManagerBuilder *unwrap(LLVMPassManagerBuilderRef P) {
                                ^
executionengine.cpp:53:27: note: return value type mismatch
     inline TargetMachine *unwrap(LLVMTargetMachineRef P) {
                           ^
/usr/lib/llvm-7/include/llvm/Target/TargetMachine.h:59:7: note: type name 'llvm::TargetMachine' should match type name 'llvm::PassManagerBuilder'
 class TargetMachine {
       ^
/usr/lib/llvm-7/include/llvm/Transforms/IPO/PassManagerBuilder.h:59:7: note: the incompatible type is defined here
 class PassManagerBuilder {
       ^
executionengine.cpp:53:27: note: 'unwrap' was previously declared here
     inline TargetMachine *unwrap(LLVMTargetMachineRef P) {
                           ^
executionengine.cpp:53:27: note: code may be misoptimized unless -fno-strict-aliasing is used
/usr/bin/ld: /tmp/ccmWo70g.ltrans0.ltrans.o: relocation R_X86_64_PC32 against symbol `_ZTVN4llvm11raw_ostreamE@@LLVM_7' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile.linux:22: libllvmlite.so] Error 1

[sklam]

How are you getting llvm7?

[danstender]

It's the package in the Debian archive [1]. Checked again, the problem appears with 0.27.0 and LLVM 7.0.1-3 [2].

[1] https://tracker.debian.org/pkg/llvm-toolchain-7

[2] https://paste.debian.net/1057905/

[aguinet]

One workaround to do this is to compile llvmlite by forcing the usage of -fPIC:

CXX="g++ -fPIC" pip install llvmlite

You might not see this on Ubuntu machines because -fPIC and -pie are set by default (https://wiki.ubuntu.com/SecurityTeam/PIE).

I guess this needs to be fixed in llvmlite's setup.py somehow!

[seibert]

I'm assuming these flags are not reported by llvm-config on Ubuntu? When we build our wrapper around LLVM, we do use llvm-config to ensure we build with the same flags that LLVM was built with.

[aguinet]

That's llvm-config-7 running from the official Xenial docker and the llvm-7 binary from apt.llvm.org:

# llvm-config-7 --cflags
-I/usr/lib/llvm-7/include -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

That's llvm-config-7 running on a Debian sid machine using the Debian sid package:

$ llvm-config-7 --cflags
-I/usr/lib/llvm-7/include -g -O2 -fdebug-prefix-map=/build/llvm-toolchain-7-H6mBL4/llvm-toolchain-7-7=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-comment -ffunction-sections -fdata-sections -O2 -g -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

-fPIC is in the second, but not the first one. That might be related to the fact that Ubuntu has -fPIC by default in its own GCC.

I'd say that, just in case, llvmlite should append this flag when compiling its extension under Linux systems!

[sylvestre]

We fixed a long standing issue in LLVM https://bugs.llvm.org/show_bug.cgi?id=8220
https://reviews.llvm.org/D55391

And I took the fix in Debian + Ubuntu + apt.llvm.org to address it in llvm-toolchain-7 (the issue became more important as we moved to a stage2 build of llvm-toolchain)
See the first item here:
https://tracker.debian.org/news/1011082/accepted-llvm-toolchain-7-1701rc3-2-source-into-unstable/

[danstender]

However adding -fPIC worked. I've patched ffi/build.py to recognize CXXFLAGS and LDFLAGS from the build environment.

[aguinet]

Thanks @sylvestre for the explanation, this makes more sense now :) And thanks @danstender for the fix!

[sylvestre]

De rien!

[danstender]

Welcome. LLVMlite is ready for Debian Buster then. But as a perspective, having this issue solved upstream would be preferable instead of keeping a delta. Would you like to include something like to recognize external env build flags if they are present/set?

[astrojuanlu]

I am getting a similar error again using llvmdev from the numba channel:

(poliastro37) juanlu@centauri ~ $ llvm-config --prefix
/home/juanlu/.miniconda36/envs/poliastro37
(poliastro37) juanlu@centauri ~ $ llvm-config --version
8.0.0
(poliastro37) juanlu@centauri ~ $ llvm-config --cflags
-I/home/juanlu/.miniconda36/envs/poliastro37/include  -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
(poliastro37) juanlu@centauri ~ $ pip install git+https://github.com/numba/llvmlite.git -v
Created temporary directory: /tmp/pip-ephem-wheel-cache-bu3u1ijz
Created temporary directory: /tmp/pip-req-tracker-w5byulge
Created requirements tracker '/tmp/pip-req-tracker-w5byulge'
Created temporary directory: /tmp/pip-install-yglj_8tf
Collecting git+https://github.com/numba/llvmlite.git
  Created temporary directory: /tmp/pip-req-build-0pq_sw9v
  Cloning https://github.com/numba/llvmlite.git to /tmp/pip-req-build-0pq_sw9v
  Running command git clone -q https://github.com/numba/llvmlite.git /tmp/pip-req-build-0pq_sw9v
  Added git+https://github.com/numba/llvmlite.git to build tracker '/tmp/pip-req-tracker-w5byulge'
    Running setup.py (path:/tmp/pip-req-build-0pq_sw9v/setup.py) egg_info for package from git+https://github.com/numba/llvmlite.git
    Running command python setup.py egg_info
    running egg_info
    creating pip-egg-info/llvmlite.egg-info
    writing pip-egg-info/llvmlite.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/llvmlite.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/llvmlite.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/llvmlite.egg-info/SOURCES.txt'
    reading manifest file 'pip-egg-info/llvmlite.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    /home/juanlu/.miniconda36/envs/poliastro37/lib/python3.7/site-packages/setuptools/dist.py:472: UserWarning: Normalizing '0.29.0dev0+30.g1038ba6' to '0.29.0.dev0+30.g1038ba6'
      normalized_version,
    warning: no previously-included files matching 'CMakeCache.txt' found anywhere in distribution
    writing manifest file 'pip-egg-info/llvmlite.egg-info/SOURCES.txt'
  Source in /tmp/pip-req-build-0pq_sw9v has version 0.29.0.dev0+30.g1038ba6, which satisfies requirement llvmlite==0.29.0.dev0+30.g1038ba6 from git+https://github.com/numba/llvmlite.git
  Removed llvmlite==0.29.0.dev0+30.g1038ba6 from git+https://github.com/numba/llvmlite.git from build tracker '/tmp/pip-req-tracker-w5byulge'
Building wheels for collected packages: llvmlite
  Created temporary directory: /tmp/pip-wheel-cc71osvt
  Building wheel for llvmlite (setup.py) ...   Destination directory: /tmp/pip-wheel-cc71osvt
  Running command /home/juanlu/.miniconda36/envs/poliastro37/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-req-build-0pq_sw9v/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-cc71osvt --python-tag cp37
  /home/juanlu/.miniconda36/envs/poliastro37/lib/python3.7/site-packages/setuptools/dist.py:472: UserWarning: Normalizing '0.29.0dev0+30.g1038ba6' to '0.29.0.dev0+30.g1038ba6'
    normalized_version,
  running bdist_wheel
  /home/juanlu/.miniconda36/envs/poliastro37/bin/python /tmp/pip-req-build-0pq_sw9v/ffi/build.py
  LLVM version... # static-libstdc++ avoids runtime dependencies on a
  # particular libstdc++ version.
  g++  -shared -I/home/juanlu/.miniconda36/envs/poliastro37/include -std=c++17 -fno-exceptions -fno-rtti -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -g -DHAVE_SVML -flto assembly.cpp bitcode.cpp core.cpp initfini.cpp module.cpp value.cpp executionengine.cpp transforms.cpp passmanagers.cpp targets.cpp dylib.cpp linker.cpp object_file.cpp -o libllvmlite.so  -L/home/juanlu/.miniconda36/envs/poliastro37/lib -flto -Wl,--exclude-libs=ALL -lLLVMFuzzMutate -lLLVMCoverage -lLLVMDlltoolDriver -lLLVMOrcJIT -lLLVMMCA -lLLVMWindowsManifest -lLLVMTextAPI -lLLVMMCJIT -lLLVMCoroutines -lLLVMLTO -lLLVMPasses -lLLVMObjCARCOpts -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMMIRParser -lLLVMTableGen -lLLVMIntelJITEvents -lLLVMDebugInfoDWARF -lLLVMXRay -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMWebAssemblyDisassembler -lLLVMWebAssemblyCodeGen -lLLVMWebAssemblyDesc -lLLVMWebAssemblyAsmPrinter -lLLVMWebAssemblyAsmParser -lLLVMWebAssemblyInfo -lLLVMNVPTXCodeGen -lLLVMNVPTXDesc -lLLVMNVPTXInfo -lLLVMNVPTXAsmPrinter -lLLVMAMDGPUDisassembler -lLLVMMCDisassembler -lLLVMAMDGPUCodeGen -lLLVMGlobalISel -lLLVMSelectionDAG -lLLVMipo -lLLVMInstrumentation -lLLVMVectorize -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser -lLLVMAsmPrinter -lLLVMAMDGPUAsmParser -lLLVMAMDGPUDesc -lLLVMAMDGPUInfo -lLLVMAMDGPUAsmPrinter -lLLVMAMDGPUUtils -lLLVMObjectYAML -lLLVMOptRemarks -lLLVMLibDriver -lLLVMOption -lLLVMLineEditor -lLLVMInterpreter -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMCodeGen -lLLVMTarget -lLLVMScalarOpts -lLLVMInstCombine -lLLVMAggressiveInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMObject -lLLVMMCParser -lLLVMMC -lLLVMDebugInfoCodeView -lLLVMDebugInfoMSF -lLLVMBitReader -lLLVMCore -lLLVMBinaryFormat -lLLVMSupport -lLLVMDemangle -lz -lrt -ldl -lpthread -lm
  /usr/bin/ld: /tmp/cc2piJCD.ltrans0.ltrans.o: relocation R_X86_64_PC32 against symbol `_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev' can not be used when making a shared object; recompile with -fPIC
  /usr/bin/ld: final link failed: Bad value
  collect2: error: ld returned 1 exit status
  Makefile.linux:20: recipe for target 'libllvmlite.so' failed
  make: *** [libllvmlite.so] Error 1
  8.0.0

  SVML detected
  Traceback (most recent call last):
    File "/tmp/pip-req-build-0pq_sw9v/ffi/build.py", line 168, in <module>
      main()
    File "/tmp/pip-req-build-0pq_sw9v/ffi/build.py", line 158, in main
      main_posix('linux', '.so')
    File "/tmp/pip-req-build-0pq_sw9v/ffi/build.py", line 150, in main_posix
      subprocess.check_call(['make', '-f', makefile])
    File "/home/juanlu/.miniconda36/envs/poliastro37/lib/python3.7/subprocess.py", line 347, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['make', '-f', 'Makefile.linux']' returned non-zero exit status 2.
  error: command '/home/juanlu/.miniconda36/envs/poliastro37/bin/python' failed with exit status 1
error
  ERROR: Failed building wheel for llvmlite
[...]