"ld.lld: error: corrupt input file: version definition index 0 for symbol curl_jmpenv is out of bounds" for libtensorflow.so

[Joshi-Tejas]

Hello,
@kshiteejm @mtrofin
I am trying to learn MLGO and would like to contribute to it. I went through the main paper and documentation. While following the Fuchsia demo, I have run into an error with libtensorflow.so:

ninja distribution
[1/1273] Linking CXX executable bin/clang-ast-dump
FAILED: bin/clang-ast-dump
: && /usr/bin/c++ -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -ffile-prefix-map=/home/amd/tejas/MLGO/ninja-build=../ninja-build -ffile-prefix-map=/home/amd/tejas/MLGO/llvm-project/= -no-canonical-prefixes -fno-common -Woverloaded-virtual -fno-strict-aliasing -O3 -DNDEBUG -static-libstdc++ -fuse-ld=lld -Wl,--color-diagnostics    -Wl,--gc-sections tools/clang/lib/Tooling/DumpTool/CMakeFiles/clang-ast-dump.dir/ASTSrcLocProcessor.cpp.o tools/clang/lib/Tooling/DumpTool/CMakeFiles/clang-ast-dump.dir/ClangSrcLocDump.cpp.o -o bin/clang-ast-dump  -Wl,-rpath,"\$ORIGIN/../lib:/home/amd/tejas/MLGO/tensorflow/lib"  lib/libLLVMOption.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMSupport.a  lib/libclangAST.a  lib/libclangASTMatchers.a  lib/libclangBasic.a  lib/libclangDriver.a  lib/libclangFrontend.a  lib/libclangSerialization.a  lib/libclangToolingCore.a  lib/libclangDriver.a  lib/libLLVMWindowsDriver.a  lib/libLLVMOption.a  lib/libclangParse.a  lib/libclangSema.a  lib/libclangEdit.a  lib/libclangAnalysis.a  lib/libclangASTMatchers.a  lib/libclangAST.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMScalarOpts.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMProfileData.a  lib/libLLVMSymbolize.a  lib/libLLVMDebugInfoPDB.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMDebugInfoDWARF.a  /home/amd/tejas/MLGO/tensorflow/lib/libtensorflow.so  /home/amd/tejas/MLGO/tensorflow/lib/libtensorflow_framework.so  lib/libLLVMObject.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMTextAPI.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libclangRewrite.a  lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMSupport.a  -lrt  -ldl  -lm  /usr/lib/x86_64-linux-gnu/libz.so  lib/libLLVMDemangle.a && :
ld.lld: warning: found local symbol 'VERS_1.0' in global part of symbol table in file /home/amd/tejas/MLGO/tensorflow/lib/libtensorflow.so
ld.lld: error: corrupt input file: version definition index 0 for symbol curl_jmpenv is out of bounds
defined in /home/amd/tejas/MLGO/tensorflow/lib/libtensorflow.so
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

I am strictly following the demo steps but my system is Ubuntu 22.04. So for python compatibility, I have created venv with Python 3.8.10 (all the python libraries and tensorflow dependencies were installed successfully). But now, I am stuck here and couldn't find any solution. Is the libtensorflow version mentioned in the demo not compatible now? I couldn't try any other as I don't know how to get it.
Thanks!

[mtrofin]

This has to do with the tensorflow C API library, not the pip package. See issue #71 , does that help?

[Joshi-Tejas]

I am actually using the 1.15 tensorflow C API library mentioned in the demo but yes, using gold instead of lld solved the issue for me. Do we have a IRC/Discord for discussion about the project (as I am facing some other issues)?

Thanks!

[mtrofin]

It'd be the llvm discord, but I (at least) am rarely there, not sure about the others; actually... looks like i'm logged on on my phone, so nevermind. I'm mtrofin there, too.

But! One advantage of talking on github issues is that it's then available/visible to others who might hit similar issues.

[Joshi-Tejas]

Okay, got it, thanks. Just as I mentioned in my last comment that I was able to solve the issue but I am now facing different issues when I try different combinations:

#######################################################################################

1. Setting llvm version with which current fuchsia was built (getting the git revision from "jiri package 'fuchsia/third_party/clang/${platform}'") and using lld:

step : ninja distribution -j32

/home/amd/tejas/MLGO/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_fuchsia.cpp:410:24: error: use of undeclared identifier '__sanitizer_get_configuration'
  zx_status_t status = __sanitizer_get_configuration(file_name, &vmo);

##########################################################################################

2. Setting llvm version with which current fuchsia was built and using gold (using gold here means commenting out LLD use from clang/cmake/caches/Fuchsia-stage2.cmake, using -DLLVM_USE_LINKER=gold while configuring and changing -fuse-ld=lld to gold):

step : ninja distribution -j32

CMake Error at /home/amd/tejas/MLGO/llvm-project/compiler-rt/cmake/config-ix.cmake:196 (message):
  Please use architecture with 4 or 8 byte pointers.
[2/2] Linking CXX executable cmTC_7c1e7
FAILED: cmTC_7c1e7
: && /home/amd/tejas/MLGO/ninja-build/./bin/clang++ --target=aarch64-unknown-linux-gnu --sysroot=/home/amd/tejas/MLGO/fuchsia-sysroot/linux-arm64 --target=aarch64-unknown-linux-gnu -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-comment -fdiagnostics-color  -nostdinc++  -isystem /home/amd/tejas/MLGO/llvm-project/llvm/runtimes/../../libcxx/include -Wl,--version-script,/home/amd/tejas/MLGO/ninja-build/runtimes/runtimes-aarch64-unknown-linux-gnu-bins/CMakeFiles/CMakeTmp/dummy.vers -fuse-ld=gold CMakeFiles/cmTC_7c1e7.dir/src.cxx.o -o cmTC_7c1e7   && :
/home/amd/tejas/MLGO/ninja-build/./bin/clang++: Relink `/home/amd/tejas/MLGO/tensorflow/lib/libtensorflow_framework.so.1' with `/lib/x86_64-linux-gnu/libz.so.1' for IFUNC symbol `crc32_z'
clang-12: warning: argument unused during compilation: '-nostdinc++' [-Wunused-command-line-argument]
/usr/bin/ld.gold: fatal error: /home/amd/tejas/MLGO/fuchsia-sysroot/linux-arm64/usr/lib/gcc/aarch64-linux-gnu/4.9/../../../aarch64-linux-gnu/crt1.o: unsupported ELF machine number 183

##############################################################################################

3. Setting llvm version to 15.x release branch, latest fuchsia and using lld:

step : ninja distribution -j32

[2994/4285] ASTNodeAPI.json
FAILED: tools/clang/lib/Tooling/ASTNodeAPI.json /home/amd/tejas/MLGO/ninja-build/tools/clang/lib/Tooling/ASTNodeAPI.json
cd /home/amd/tejas/MLGO/ninja-build/tools/clang/lib/Tooling && /home/amd/tejas/MLGO/ninja-build/bin/clang-ast-dump --skip-processing=0 -I /home/amd/tejas/MLGO/ninja-build/lib/clang/15.0.0/include -I /home/amd/tejas/MLGO/llvm-project/clang/include -I /home/amd/tejas/MLGO/ninja-build/tools/clang/include -I /home/amd/tejas/MLGO/ninja-build/include -I /home/amd/tejas/MLGO/llvm-project/llvm/include -I /usr/include/c++/11 -I /usr/include/x86_64-linux-gnu/c++/11 -I /usr/include/c++/11/backward -I /usr/lib/gcc/x86_64-linux-gnu/11/include -I /usr/local/include -I /usr/include/x86_64-linux-gnu -I /usr/include --json-output-path /home/amd/tejas/MLGO/ninja-build/tools/clang/lib/Tooling/ASTNodeAPI.json
free(): invalid pointer
Aborted (core dumped)

#################################################################################################

4. Setting llvm version to 15.x release branch and using gold (using gold here means commenting out LLD use from clang/cmake/caches/Fuchsia-stage2.cmake, using -DLLVM_USE_LINKER=gold while configuring and changing -fuse-ld=lld to gold):

step : ninja distribution -j32

CMake Error at /home/amd/tejas/MLGO/llvm-project/llvm/cmake/modules/HandleLLVMOptions.cmake:309 (message):
  Host compiler does not support '-fuse-ld=gold'
Call Stack (most recent call first):
  CMakeLists.txt:149 (include)

Run Build Command(s):/usr/bin/ninja cmTC_858e7 && [1/2] Building CXX object CMakeFiles/cmTC_858e7.dir/src.cxx.obj
clang-15: warning: argument unused during compilation: '-fuse-ld=gold' [-Wunused-command-line-argument]
[2/2] Linking CXX executable cmTC_858e7
FAILED: cmTC_858e7
: && /home/amd/tejas/MLGO/ninja-build/./bin/clang++ --target=aarch64-unknown-fuchsia --sysroot=/home/amd/tejas/MLGO/fuchsia-idk/arch/arm64/sysroot --target=aarch64-unknown-fuchsia -I/home/amd/tejas/MLGO/fuchsia-idk/pkg/sync/include -I/home/amd/tejas/MLGO/fuchsia-idk/pkg/fdio/include  --start-no-unused-arguments --unwindlib=none --end-no-unused-arguments -nostdlib++ -nostdinc++ -fuse-ld=gold -L/home/amd/tejas/MLGO/fuchsia-idk/arch/arm64/lib CMakeFiles/cmTC_858e7.dir/src.cxx.obj -o cmTC_858e7   && :
clang-15: warning: argument unused during compilation: '-nostdinc++' [-Wunused-command-line-argument]
/usr/bin/ld.gold: fatal error: /home/amd/tejas/MLGO/fuchsia-idk/arch/arm64/sysroot/lib/Scrt1.o: unsupported ELF machine number 183

Can you suggest a combination which works for you?
Thanks!

[mtrofin]

Have you tried the docker file mentioned in #71 ?

[Joshi-Tejas]

Hey, the Dockerfile worked with very minor changes. I still wonder why it failed for me but anyways, thanks! Looking forward to contribute!

[mtrofin]

Awesome!