[LLVM] Can't compile bash 5 using the toolchain

[mikehearn]

Using GraalVM EE 20.1-dev on macOS High Sierra

bash-3.2$ java -Xinternalversion
OpenJDK 64-Bit Server VM (11.0.6+9-jvmci-20.0-b02) for bsd-amd64 JRE (11.0.6+9-jvmci-20.0-b02), built on Jan 20 2020 14:32:51 by "graal1" with gcc 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)

bash-3.2$ gcc --version
clang version 9.0.0 (GraalVM.org llvmorg-9.0.0-5-g80b1d876fd-bgb66b241662 80b1d876fd4296b48433de5b66eaebe551897508)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Users/mike/JVMs/graalvm-ce-java11-20.1.0-dev/Contents/Home/lib/llvm/bin

I tried building bash 5.0 with the LLVM toolchain. It fails with this error in the final link step:

gcc -L./builtins -L./lib/readline -L./lib/readline -L./lib/glob -L./lib/tilde  -L./lib/sh   -g -O2 -Wno-parentheses -Wno-format-security   -o bash shell.o eval.o y.tab.o general.o make_cmd.o print_cmd.o  dispose_cmd.o execute_cmd.o variables.o copy_cmd.o error.o expr.o flags.o jobs.o subst.o hashcmd.o hashlib.o mailcheck.o trap.o input.o unwind_prot.o pathexp.o sig.o test.o version.o alias.o array.o arrayfunc.o assoc.o braces.o bracecomp.o bashhist.o bashline.o  list.o stringlib.o locale.o findcmd.o redir.o pcomplete.o pcomplib.o syntax.o xmalloc.o  -lbuiltins -lglob -lsh -lreadline -lhistory -ltermcap -ltilde  lib/intl/libintl.a -liconv -liconv  -ldl
duplicate symbol '_get_history_event' in:
    ./lib/readline/libreadline.a(histexpand.o)
    ./lib/readline/libhistory.a(histexpand.o)
duplicate symbol '_where_history' in:
    ./lib/readline/libreadline.a(history.o)
    ./lib/readline/libhistory.a(history.o)
... etc ....

It builds fine using a regular macOS toolchain.

Steps to reproduce the problem

1. Download bash-5.0.tar.gz source code from GNU.
2. Set PATH to be prefixed with the LLVM toolchain directory (languages/llvm/native/bin) and the GraalVM java bin directory.
3. Run ./configure && make

Investigation

Both these .a files do indeed contain the same .o files exporting the same symbols. This is true of a successful normal build so it seems to be either intentional or harmless usually, but the LLVM toolchain doesn't seem to tolerate it.

[zapster]

Hi @mikehearn!

Thanks for giving the toolchain a try! I was able to reproduce the issues with the latest version so did a little digging. Turns out that the issue can be reproduced without using the GraalVM LLVM toolchain. If I compile bash using llvm-ar and llvm-ranlib (which is what our toolchain uses under the hood), I run into the same issue:

$ AR=/path/to/llvm_build/bin/llvm-ar RANLIB=/path/to/llvm_build/bin/llvm-ranlib ../configure
$ make -j
...
gcc -L./builtins -L./lib/readline -L./lib/readline -L./lib/glob -L./lib/tilde  -L./lib/sh  -Wl,-why_load  -g -O2 -Wno-parentheses -Wno-format-security   -o bash shell.o eval.o y.tab.o general.o make_cmd.o print_cmd.o  dispose_cmd.o execute_cmd.o variables.o copy_cmd.o error.o expr.o flags.o jobs.o subst.o hashcmd.o hashlib.o mailcheck.o trap.o input.o unwind_prot.o pathexp.o sig.o test.o version.o alias.o array.o arrayfunc.o assoc.o braces.o bracecomp.o bashhist.o bashline.o  list.o stringlib.o locale.o findcmd.o redir.o pcomplete.o pcomplib.o syntax.o xmalloc.o  -lbuiltins -lglob -lsh -lreadline -lhistory -ltermcap -ltilde  lib/intl/libintl.a -liconv -liconv  -ldl
duplicate symbol _copy_history_entry in:
    ./lib/readline/libreadline.a(history.o)
    ./lib/readline/libhistory.a(history.o)
duplicate symbol _free_history_entry in:
    ./lib/readline/libreadline.a(history.o)
    ./lib/readline/libhistory.a(history.o)
duplicate symbol _replace_history_entry in:
    ./lib/readline/libreadline.a(history.o)
    ./lib/readline/libhistory.a(history.o)
...
ld: 38 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bash] Error 1

I filled an issue for LLVM. Let's hope that they can help.

In the meanwhile, you can work around the issue by manually deleting the duplicate object files from one of the archives, e.g.:

$ ar d ./lib/readline/libreadline.a history.o histexpand.o

After doing that, make should succeed. Also, see my word of caution in the related issue #2301 (comment).

[zapster]

closing as it is/was not a graalvm problem. the upstream ticket is not yet closed, but it might be worth trying a newer llvm version. maybe the issue was fixed in the mean time