Build Julia1.0-compatible Clang/LLVM binaries #34

Gnimuc · 2018-09-18T15:58:07Z

As per discussed on Slack, there are no versions of Clang/LLVM binaries in the LLVMBuilder that are being compatible with the LLVM shipped with Julia 1.0. For example, loading and using libclang.dylib(6.0.0-6) will crash Julia.
It could work as long as we rebuild Julia with USE_BINARYBUILDER_LLVM=1(use the LLVMBuilder binaries). However, since Julia 1.0 has already been shipped, for compatibility's sake, we may need to tweak LLVMBuilder for building a version that can work along with Julia 1.0.

current status(updated)	Julia 1.0 (LLVM 6.0.0)	Julia 1.x (LLVM 6.0.1)
LLVM original	❌	❌
LLVM w/ WebAssembly	❌	✅
LLVM w/o WebAssembly	✅	✅

cc @staticfloat

julia> versioninfo()
Julia Version 1.0.0
Commit 5d4eaca0c9 (2018-08-08 20:58 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = atom
  JULIA_NUM_THREADS = 4

julia> using Libdl
julia> clang = Libdl.dlopen_e("./libclang.dylib")  # 6.0.0-6
julia> idx = Libdl.dlsym_e(clang, :clang_createIndex)
julia> ccall(idx, Ptr{Cvoid}, (Int32, Int32), 1, 1)
dyld: lazy symbol binding failed: Symbol not found: _LLVMInitializeWebAssemblyTargetInfo
  Referenced from: ./libclang.dylib
  Expected in: /Applications/Julia-1.0.app/Contents/Resources/julia/lib/julia/libLLVM.dylib

dyld: Symbol not found: _LLVMInitializeWebAssemblyTargetInfo
  Referenced from: ./libclang.dylib
  Expected in: /Applications/Julia-1.0.app/Contents/Resources/julia/lib/julia/libLLVM.dylib


signal (6): Abort trap: 6
in expression starting at nin expression starting at no file:0
in expression starting at no file:0
unknown function (ip: 0x10636b249)
Allocations: 13090829 (Pool: 13088353; Big: 2476); GC: 28
[1]    16239 abort      julia

julia> versioninfo()
Julia Version 1.1.0-DEV.279
Commit 3a15e6bd5b (2018-09-18 01:19 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin17.7.0)
  CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = atom
  JULIA_NUM_THREADS = 4

julia> using Libdl

julia> clang = Libdl.dlopen_e("./libclang.dylib")
Ptr{Nothing} @0x00007fd706a308b0

julia> idx = Libdl.dlsym_e(clang, :clang_createIndex)
Ptr{Nothing} @0x0000000122c3ef90

julia> ccall(idx, Ptr{Cvoid}, (Int32, Int32), 1, 1)
Ptr{Nothing} @0x00007fd704417c90

The text was updated successfully, but these errors were encountered:

vchuravy · 2018-09-19T16:03:14Z

In a Julia 1.0 compatible build we need to turn off:

WebAssembly

That is the only one that immediately jumps at me. We would also need to turn off Polly, but that is already the case.

An alternative solution that we might try first is to set:

LLVMBuilder/patches/llvm_patches/9999-llvm-symver-jlprefix.patch

Line 16 in 5708d3d

+JL_LLVM_@LLVM_VERSION_MAJOR@.@LLVM_VERSION_MINOR@ { global: *; };

To something else aka JLBB_LLVM_ instead of JL_LLVM.

@Gnimuc could you give that a try?

Gnimuc · 2018-09-21T13:49:19Z

@vchuravy just built 6.0.0-6(d1ae9ac) without WebAssembly enabled. Although there is a warning, the libclang works pretty well. I also tried the second solution with the latest master, but no luck: https://gist.github.com/Gnimuc/f726f7f2ae8185f256723784b34756ac

make release
julia build_tarballs.jl
[ Info: Building for x86_64-linux-musl
[ Info: Building for x86_64-linux-gnu
==29021==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
Warning: lib/clang/6.0.0/lib/linux/libclang_rt.asan-x86_64.so cannot be dlopen()'ed

julia> using Libdl

julia> clang = Libdl.dlopen_e("./libclang.so")
Ptr{Nothing} @0x0000000001382df0

julia> idx = Libdl.dlsym_e(clang, :clang_createIndex)
Ptr{Nothing} @0x00007f8cf783f1e0

julia> ccall(idx, Ptr{Cvoid}, (Int32, Int32), 1, 1)
Ptr{Nothing} @0x000000000145bb80

julia> versioninfo()
Julia Version 1.0.0
Commit 5d4eaca0c9 (2018-08-08 20:58 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, haswell)

Gnimuc · 2018-09-22T01:10:32Z

~~failed to build for LLVM on x86_64-w64-mingw32~~

[100%] Linking CXX executable ../../../../bin/clang.exe
cd /workspace/srcdir/llvm-6.0.0.src/build/tools/clang/tools/driver && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/clang.dir/link.txt --verbose=1
/usr/local/bin/cmake -E remove -f CMakeFiles/clang.dir/objects.a
/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-ar cr CMakeFiles/clang.dir/objects.a @CMakeFiles/clang.dir/objects1.rsp
/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-g++ --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/  -remap -D__USING_SJLJ_EXCEPTIONS__ -D__CRT__NO_INLINE -Wa,-mbig-obj -Werror=date-time -std=gnu++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wundef -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fno-common -Woverloaded-virtual -fno-strict-aliasing  -O2 -DNDEBUG   -Wl,--stack,16777216 -Wl,--whole-archive CMakeFiles/clang.dir/objects.a -Wl,--no-whole-archive  -o ../../../../bin/clang.exe -Wl,--out-implib,../../../../lib/libclang.dll.a -Wl,--major-image-version,0,--minor-image-version,0 @CMakeFiles/clang.dir/linklibs.rsp
[100%] Linking CXX executable ../../../../bin/clang-import-test.exe
cd /workspace/srcdir/llvm-6.0.0.src/build/tools/clang/tools/clang-import-test && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/clang-import-test.dir/link.txt --verbose=1
/usr/local/bin/cmake -E remove -f CMakeFiles/clang-import-test.dir/objects.a
/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-ar cr CMakeFiles/clang-import-test.dir/objects.a @CMakeFiles/clang-import-test.dir/objects1.rsp
/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-g++ --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/  -remap -D__USING_SJLJ_EXCEPTIONS__ -D__CRT__NO_INLINE -Wa,-mbig-obj -Werror=date-time -std=gnu++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wundef -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fno-common -Woverloaded-virtual -fno-strict-aliasing  -O2 -DNDEBUG   -Wl,--stack,16777216 -Wl,--whole-archive CMakeFiles/clang-import-test.dir/objects.a -Wl,--no-whole-archive  -o ../../../../bin/clang-import-test.exe -Wl,--out-implib,../../../../lib/libclang-import-test.dll.a -Wl,--major-image-version,0,--minor-image-version,0 @CMakeFiles/clang-import-test.dir/linklibs.rsp
[100%] Linking CXX executable ../../../../bin/c-index-test.exe
cd /workspace/srcdir/llvm-6.0.0.src/build/tools/clang/tools/c-index-test && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/c-index-test.dir/link.txt --verbose=1
/usr/local/bin/cmake -E remove -f CMakeFiles/c-index-test.dir/objects.a
/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-ar cr CMakeFiles/c-index-test.dir/objects.a @CMakeFiles/c-index-test.dir/objects1.rsp
/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-g++ --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/  -remap -D__USING_SJLJ_EXCEPTIONS__ -D__CRT__NO_INLINE -Wa,-mbig-obj -Werror=date-time -std=gnu++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wundef -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fno-common -Woverloaded-virtual -fno-strict-aliasing  -O2 -DNDEBUG   -Wl,--stack,16777216 -Wl,--whole-archive CMakeFiles/c-index-test.dir/objects.a -Wl,--no-whole-archive  -o ../../../../bin/c-index-test.exe -Wl,--out-implib,../../../../lib/libc-index-test.dll.a -Wl,--major-image-version,0,--minor-image-version,0 @CMakeFiles/c-index-test.dir/linklibs.rsp
../../../../lib/libclangCodeGen.a(CGObjCMac.cpp.obj):CGObjCMac.cpp:(.text$_ZN5clang7CodeGen13CGObjCRuntime29EmitNSAutoreleasePoolClassRefERNS0_15CodeGenFunctionE+0x0): multiple definition of `clang::CodeGen::CGObjCRuntime::EmitNSAutoreleasePoolClassRef(clang::CodeGen::CodeGenFunction&)'
../../../../lib/libclangCodeGen.a(CGObjC.cpp.obj):CGObjC.cpp:(.text$_ZN5clang7CodeGen13CGObjCRuntime29EmitNSAutoreleasePoolClassRefERNS0_15CodeGenFunctionE+0x0): first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [tools/clang/tools/driver/CMakeFiles/clang.dir/build.make:143: bin/clang.exe] Error 1
make[2]: Leaving directory '/workspace/srcdir/llvm-6.0.0.src/build'
make[1]: *** [CMakeFiles/Makefile2:15803: tools/clang/tools/driver/CMakeFiles/clang.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/workspace/srcdir/llvm-6.0.0.src/build'
[100%] Built target clang-import-test
make[2]: Leaving directory '/workspace/srcdir/llvm-6.0.0.src/build'
[100%] Built target c-index-test
make[1]: Leaving directory '/workspace/srcdir/llvm-6.0.0.src/build'
make: *** [Makefile:152: all] Error 2
ERROR: LoadError: Build for LLVM on x86_64-w64-mingw32 did not complete successfully

Gnimuc · 2018-09-22T04:48:32Z

oh, never mind. I gave it another try and this time, no errors. I've tested the first solution on Linux(x64), Windows(x86+x64) and MacOS, all works great!

staticfloat · 2018-09-22T07:51:16Z

That’s awesome! To be clear, this is with your custom-built Julia or the official Julia 1.0 binaries?

Gnimuc · 2018-09-22T07:53:46Z

All of the tests are on the official Julia 1.0 binaries. (Never thought this issue could be one-line fixed!)

vchuravy · 2018-09-22T13:59:05Z

@tshort how is your long-term need for WebAssembly support? If you want it we should enable it in Julia base. I should also note that with LLVM 8 WebAssembly most likely will be promoted to an officially supported backend and not an experimental one.

…

On Sat, Sep 22, 2018, 03:53 Gnimuc ***@***.***> wrote: All of the tests are on the official Julia 1.0 binaries. (Never thought this issue could be one-line fixed!) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAI3agXVK2VFLNokBFP_NfaNAlR5iHXuks5udeyKgaJpZM4WuPxi> .

Gnimuc · 2018-09-24T06:37:46Z

Would you mind tagging a new v6.0.0-7 release with WebAssembly disabled?

staticfloat · 2018-09-24T12:09:34Z

I'm going to tag a v6.0.1-3+nowasm build that will stay on its own branch. God willing, we will eventually be able to merge the two, but some people need WebAssembly, so we'll keep them separate for now.

vchuravy · 2018-09-24T12:15:10Z

Instead of having a nowasm tag we should just add binary artefacts.

…

On Mon, Sep 24, 2018, 08:09 Elliot Saba ***@***.***> wrote: I'm going to tag a v6.0.1-3+nowasm build that will stay on its own branch. God willing, we will eventually be able to merge the two, but some people need WebAssembly, so we'll keep them separate for now. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAI3am9XscqABW2TqwCgYCs4ROxzXPuOks5ueMt-gaJpZM4WuPxi> .

staticfloat · 2018-09-24T17:21:04Z

I talked it over with Valentine, and for now we're just going to do a separate tag, as it appears likely that in the future (e.g. for Julia 1.1+) we will be able to have compatibility with Julia, even with WASM turned on: JuliaLang/julia#29341

1. use BinaryProvider instead of Homebrew on MacOS 2. Simplify build script on Linux 3. Add Windows support

staticfloat · 2018-10-18T01:05:02Z

https://github.com/staticfloat/LLVMBuilder/releases/tag/v6.0.1-3%2Bnowasm

Gnimuc · 2018-10-20T06:13:54Z

the binaries are still not dlopen-able on Windows, I'm bisecting the issue:

julia> versioninfo()
Julia Version 1.0.0
Commit 5d4eaca0c9 (2018-08-08 20:58 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

julia> using Libdl

julia> clang = Libdl.dlopen("C:\\Users\\gnimuc\\.julia\\packages\\Clang\\hbmlo\\deps\\usr\\bin\\libclang.dll")
ERROR: could not load library "C:\Users\gnimuc\.julia\packages\Clang\hbmlo\deps\usr\bin\libclang.dll"
The specified procedure could not be found.

Stacktrace:
 [1] dlopen(::String, ::UInt32) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Libdl\src\Libdl.jl:97 (repeats 2 times)
 [2] top-level scope at none:0

Gnimuc · 2018-10-21T10:56:33Z

It looks like some basic libs are not compatible:

Gnimuc · 2018-10-21T14:07:02Z

now I find even the linux version can be loaded successfully, some function calls refuse to work(throw weird errors or crash Julia). but the macos version works fine, very strange.

/home/parallels/Documents/julia/bin/julia: relocation error: /home/parallels/.julia/packages/Clang/5S3Da/deps/usr/lib/libclang.so: symbol _ZN4llvm27install_fatal_error_handlerEPFvPvRKSsbES0_ version JL_LLVM_6.0 not defined in file libLLVM-6.0.so with link time reference

staticfloat · 2018-10-23T13:19:54Z

Did you mean the linux version can not be loaded successfully?

Gnimuc · 2018-10-24T01:38:13Z

@staticfloat It can be dlopen-ed, but calling some functions got failed. I also rebuilt LLVM 6.0.0 yesterday, the result is the same(see below). ~~I'm attempted to rebuild the binaries using an older version of BinaryBuilder, but could not download the old rootfs.~~ I suspect this is a BinaryBuilder issue, is it possible to get the older rootfs before this update JuliaPackaging/BinaryBuilder.jl#355 ?

julia> using Libdl

julia> clang = Libdl.dlopen("./libclang.so")
Ptr{Nothing} @0x000000000270abb0

julia> idx = Libdl.dlsym_e(clang, :clang_createIndex)
Ptr{Nothing} @0x00007f4040ef94d0

julia> ccall(idx, Ptr{Cvoid}, (Int32, Int32), 1, 1)
julia: relocation error: ./libclang.so: symbol _ZN4llvm27install_fatal_error_handlerEPFvPvRKSsbES0_ version JL_LLVM_6.0 not defined in file libLLVM-6.0.so with link time reference

Gnimuc · 2018-10-25T13:32:26Z

oh, never mind. I did a fresh installation of an older version of BinaryBuilder(e973095) and it's working again.

julia> using Libdl

julia> clang = Libdl.dlopen_e("./libclang.so")  # 6.0.0-6
Ptr{Nothing} @0x0000000000ce5330

julia> idx = Libdl.dlsym_e(clang, :clang_createIndex)
Ptr{Nothing} @0x00007f75aec731e0

julia> ccall(idx, Ptr{Cvoid}, (Int32, Int32), 1, 1)
Ptr{Nothing} @0x0000000000caadf0

vchuravy · 2018-11-24T23:25:18Z

@staticfloat I just noticed that Julia 1.0.X will use LLVM 6.0.0 and the nowasm build we provide are for LLVM 6.0.1

staticfloat · 2018-11-25T05:28:03Z

Is there not enough API stability between 6.0.0 and 6.0.1 that this is okay?

vchuravy · 2018-11-25T21:43:03Z

Not really guaranteed by anyone... Especially since libraries like clang will resolve their weekly linked symbols to the first darn LLVM they can find.

…

On Sun, Nov 25, 2018, 00:28 Elliot Saba ***@***.*** wrote: Is there not enough API stability between 6.0.0 and 6.0.1 that this is okay? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAI3arD3CQhBHe6IpIcHminLgkxlc9z7ks5uyipjgaJpZM4WuPxi> .

Gnimuc mentioned this issue Sep 19, 2018

Make Cxx.jl ready for production environments post Julia 1.0.0 JuliaInterop/Cxx.jl#390

Closed

Gnimuc referenced this issue in JuliaInterop/Clang.jl Oct 11, 2018

Improve build scripts

3e886ba

1. use BinaryProvider instead of Homebrew on MacOS 2. Simplify build script on Linux 3. Add Windows support

Gnimuc mentioned this issue Jan 3, 2019

ERROR: LoadError: UndefVarError: JULIA_HOME not defined JuliaInterop/Cxx.jl#398

Closed

Gnimuc mentioned this issue Mar 9, 2019

Should the default platform key abi for Windows be gcc7? JuliaPackaging/BinaryProvider.jl#152

Closed

Gnimuc closed this as completed Mar 23, 2019

Gnimuc mentioned this issue Apr 12, 2019

Upgrade build system to BB2 -- Part 1 JuliaInterop/Cxx.jl#406

Merged

Gnimuc mentioned this issue Feb 16, 2020

Fix LLVM_jll's deps and compat info JuliaRegistries/General#9542

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build Julia1.0-compatible Clang/LLVM binaries #34

Build Julia1.0-compatible Clang/LLVM binaries #34

Gnimuc commented Sep 18, 2018 •

edited

Loading

vchuravy commented Sep 19, 2018

Gnimuc commented Sep 21, 2018

Gnimuc commented Sep 22, 2018 •

edited

Loading

Gnimuc commented Sep 22, 2018

staticfloat commented Sep 22, 2018

Gnimuc commented Sep 22, 2018

vchuravy commented Sep 22, 2018 via email

Gnimuc commented Sep 24, 2018

staticfloat commented Sep 24, 2018

vchuravy commented Sep 24, 2018 via email

staticfloat commented Sep 24, 2018

staticfloat commented Oct 18, 2018

Gnimuc commented Oct 20, 2018

Gnimuc commented Oct 21, 2018

Gnimuc commented Oct 21, 2018 •

edited

Loading

staticfloat commented Oct 23, 2018

Gnimuc commented Oct 24, 2018 •

edited

Loading

Gnimuc commented Oct 25, 2018

vchuravy commented Nov 24, 2018

staticfloat commented Nov 25, 2018

vchuravy commented Nov 25, 2018 via email

Build Julia1.0-compatible Clang/LLVM binaries #34

Build Julia1.0-compatible Clang/LLVM binaries #34

Comments

Gnimuc commented Sep 18, 2018 • edited Loading

vchuravy commented Sep 19, 2018

Gnimuc commented Sep 21, 2018

Gnimuc commented Sep 22, 2018 • edited Loading

Gnimuc commented Sep 22, 2018

staticfloat commented Sep 22, 2018

Gnimuc commented Sep 22, 2018

vchuravy commented Sep 22, 2018 via email

Gnimuc commented Sep 24, 2018

staticfloat commented Sep 24, 2018

vchuravy commented Sep 24, 2018 via email

staticfloat commented Sep 24, 2018

staticfloat commented Oct 18, 2018

Gnimuc commented Oct 20, 2018

Gnimuc commented Oct 21, 2018

Gnimuc commented Oct 21, 2018 • edited Loading

staticfloat commented Oct 23, 2018

Gnimuc commented Oct 24, 2018 • edited Loading

Gnimuc commented Oct 25, 2018

vchuravy commented Nov 24, 2018

staticfloat commented Nov 25, 2018

vchuravy commented Nov 25, 2018 via email

Gnimuc commented Sep 18, 2018 •

edited

Loading

Gnimuc commented Sep 22, 2018 •

edited

Loading

Gnimuc commented Oct 21, 2018 •

edited

Loading

Gnimuc commented Oct 24, 2018 •

edited

Loading