libjulia_jll 1.7

[fingolfin]

This is work in progress; I figure there'll be problems, but we
will need this to rebuild CxxWrap for 1.7 (otherwise everything
using CxxWrap will segfault there)

[fingolfin]

@giordano @staticfloat @barche So this PR verifies what was said on Slack: that to build libjulia_jll 1.7, we either need to run Julia 1.7 on Yggdrasil, or else need some other new infrastructure that allows e.g. installing LLVM JLLs for 1.7 from Julia 1.6.

So, how do we proceed? Anything I / "we" can do to help expedite this?

We need this for a new Libjulia, and I am getting more and more reports by people who tried 1.7-beta2 and found that they couldn't use "my" packages (all boiling down to the CxxWrap issues).

[fingolfin]

Note that I also tried to just build without any dependencies, on the theory that Julia builds this way on my machine, too, and is normally able to download most of what it needs. That didn't work, though, it failed compiling the Julia kernel because of some LLVm headers it couldn't find, see this log.

But I wonder if this approach could be made to work anyway. Of course it's not nice to not declare all those JLLs as dependencies, but if it would allow us to get a first version of libjulia_jll 1.7 out, that would be enough to get going with CxxWrap etc.; and a "proper" libjulia_jll 1.7 could be done later.

[vchuravy]

@fingolfin I can't edit that part on the web, can you adda another case here: https://github.com/JuliaPackaging/Yggdrasil/pull/3214/files#diff-29b8d08819b1a971a0a4f552117850f0569b1947bc3390d698455c6e3b7557b7R106 to use LLVM12?

[vchuravy]

Yay!

[fingolfin]

nice. Of course question remains whether it works. Also not optimal to call the prerelease / beta binary 1.7.0, but hey...

I'll clean up the whole thing a bit.

[fingolfin]

Also will re-enable the other platforms, we'll see...

[fingolfin]

• macOS ARM failure seems to be caused by utf8proc_jll not having a binary for that platform
• macOS Intel errors out with ld: library not found for -losxunwind
• Linux armv6l: fails with errorsrc/flisp/string.c:22:10: fatal error: utf8proc.h: No such file or directory

[giordano]

aarch64-apple-darwin and armv6l-* are basically the same issue, they are the "experimental" platforms. For macOS I think we use llvm unwind now?

[fingolfin]

Trying to fix the x86 macos build now.

Regarding the "experimental" arm platforms, should I just disable them (for now), so we can move forward?

[fingolfin]

Hmm, seems Yggdrasil doesn't want to build my latest pushes? Don't see them under https://dev.azure.com/JuliaPackaging/Yggdrasil/_build either

[giordano]

Yes, also #3251 wasn't built on master (CC: @maleadt). Agents appear to be online, I think there is something wrong with Azure Pipelines in general

[fingolfin]

Wonder if this is related: https://status.dev.azure.com/_event/246338456

[giordano]

I saw that, but I don't think we use artifacts, also, that started 12 days ago, so I feel like it's unrelated

[giordano]

I manually triggered https://dev.azure.com/JuliaPackaging/Yggdrasil/_build/results?buildId=12023&view=results for CUDA and this is now working. Let's see whether closing this pull request does the trick

[giordano]

/AzurePipelines run

[giordano]

Well, I don't know if it was thanks to me, but now everything started working.

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[fingolfin]

ERROR: LoadError: KeyError: key UUID("47c5dbc3-30ba-59ef-96a6-123e260183d9") not found

huh?

[giordano]

@vchuravy @staticfloat can some of you please look into this? I don't have access 🙈

[fingolfin]

That UUID is LLVMLibUnwind_jll which for some reason is not found? I've disabled this again for now. But then still need a way to fix the macOS Intel build.

[giordano]

Can you please point me directly to the log of the offending job? 🙂

[benlorenz]

The error comes from something like this running in julia 1.6 (there is a platform argument in the code but that doesn't make a difference):

julia> ctx = Pkg.Types.Context(;julia_version=v"1.7.0");

julia> Pkg.add(ctx,[PackageSpec(name="LLVMLibUnwind_jll")])
   Resolving package versions...
ERROR: KeyError: key UUID("47c5dbc3-30ba-59ef-96a6-123e260183d9") not found
Stacktrace:
  [1] getindex
    @ ./dict.jl:482 [inlined]
  [2] deps_graph(ctx::Pkg.Types.Context, uuid_to_name::Dict{UUID, String}, reqs::Dict{UUID, Pkg.Types.VersionSpec}, fixed::Dict{UUID, Pkg.Resolve.Fixed})
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:474
  [3] resolve_versions!(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:404
  [4] targeted_resolve(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, preserve::Pkg.Types.PreserveLevel)
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1210
  [5] tiered_resolve(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1182
  [6] _resolve
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1216 [inlined]
  [7] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Vector{UUID}; preserve::Pkg.Types.PreserveLevel, platform::Platform)
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1231
  [8] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Platform, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:203
  [9] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:154
 [10] top-level scope
    @ REPL[60]:1

LLVMLibUnwind_jll is a stdlib from the future for julia 1.6 and some of the code seems unable to deal with that (and it has changed quite a bit between 1.6 and current Pkg.jl master).

[fingolfin]

Hrm...
ERROR: LoadError: invalid git HEAD (reference 'refs/heads/master' not found)

[fingolfin]

Off-topic: in the macOS build, there are lots of warnings make: xcode-select: No such file or directory caused by the build system invoking xcode-select -p. I wonder if this should be either patched out, or if there should be a fake xcode-select binary which returns something sensible?

[fingolfin]

The macOS x86 build now proceeds a bit further, but dies like in this log

[20:03:43]  x86_64-apple-darwin14-clang++ -mmacosx-version-min=10.10 -m64 -shared  -pipe -fPIC -fno-rtti -pedantic -std=c++14 -D_FILE_OFFSET_BITS=64 -I/workspace/destdir/include -DNDEBUG  -O3 -g -D_GNU_SOURCE -I. -I/workspace/srcdir/julia-b570546b68/src -I/workspace/srcdir/julia-b570546b68/src/flisp -I/workspace/srcdir/julia-b570546b68/src/support -I/workspace/destdir/include -I/workspace/srcdir/julia-b570546b68/usr/include -DLIBRARY_EXPORTS -I/workspace/srcdir/julia-b570546b68/deps/valgrind -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fvisibility=hidden -fno-common -Wno-comment -Wpointer-arith -Wundef -DJL_BUILD_ARCH='"x86_64"' -DJL_BUILD_UNAME='"Darwin"' -I -DLLVM_SHLIB "-DJL_SYSTEM_IMAGE_PATH=\"../lib/julia/sys.dylib\"" "-DJL_LIBJULIA_SONAME=\"libjulia.1.dylib\""       "-DJL_LIBJULIA_INTERNAL_SONAME=\"libjulia-internal.1.dylib\"" ./jloptions.o ./runtime_ccall.o ./rtutils.o ./codegen.o ./llvm-ptls.o ./jltypes.o ./gf.o ./typemap.o ./smallintset.o ./ast.o ./builtins.o ./module.o ./interpreter.o ./symbol.o ./dlload.o ./sys.o ./init.o ./task.o ./array.o ./dump.o ./staticdata.o ./toplevel.o ./jl_uv.o ./datatype.o ./simplevector.o ./runtime_intrinsics.o ./precompile.o ./threading.o ./partr.o ./stackwalk.o ./gc.o ./gc-debug.o ./gc-pages.o ./gc-stacks.o ./method.o ./jlapi.o ./signal-handling.o ./safepoint.o ./timing.o ./subtype.o ./crc32c.o ./APInt-C.o ./processor.o ./ircode.o ./opaque_closure.o ./jitlayers.o ./aotcompile.o ./debuginfo.o ./disasm.o ./llvm-simdloop.o ./llvm-muladd.o ./llvm-final-gc-lowering.o ./llvm-pass-helpers.o ./llvm-late-gc-lowering.o ./llvm-lower-handlers.o ./llvm-gc-invariant-verifier.o ./llvm-propagate-addrspaces.o ./llvm-multiversioning.o ./llvm-alloc-opt.o ./cgmemmgr.o ./llvm-api.o ./llvm-remove-addrspaces.o ./llvm-remove-ni.o ./llvm-julia-licm.o ./llvm-demote-float16.o -Wl,-rpath,'@loader_path/julia/' -Wl,-rpath,'@loader_path/' -o /workspace/srcdir/julia-b570546b68/usr/lib/libjulia-internal.1.7.dylib -L/workspace/destdir/lib -Wl,-compatibility_version,1 -Wl,-current_version,1.7.0 -L/workspace/srcdir/julia-b570546b68/usr/lib -L/workspace/srcdir/julia-b570546b68/usr/lib -Xlinker -all_load ./flisp/libflisp.a -Xlinker -all_load ./support/libsupport.a -ljulia /workspace/destdir/lib/libuv.a -lutf8proc   -L/workspace/destdir/lib -lLLVM -framework CoreFoundation 
[20:03:43] dsymutil /workspace/srcdir/julia-b570546b68/usr/lib/libjulia-internal.1.7.dylib
[20:03:44] /opt/bin/x86_64-apple-darwin14-libgfortran4-cxx11-julia_version+1.7.0/dsymutil: line 5:  4128 Segmentation fault      "$@"
[20:03:44] make[1]: *** [Makefile:302: /workspace/srcdir/julia-b570546b68/usr/lib/libjulia-internal.1.7.dylib] Error 139

[giordano]

e5f6b33 (#1987)

[fingolfin]

@giordano ah thanks -- that patch did not apply anylong and I removed it, thinking "I'll see what happens if I do that", and then forgot about this when this PR took too long ;-). I've added a new equivalent patch, this time with a git commit message in it (borrowed from your PR) to indicate what it is about.

[fingolfin]

Seems all is building. However, there are a bunch of warnings that may need to be resolved. Also, I am not sure those Dependencies, as they are, are "right" / "what we want" ? Perhaps @vchuravy @giordano @staticfloat would you like to scrutinize this some more.

[vchuravy]

Looks good from my side.

The only question I had is why we don't list the version explicitly per dependency. E.g. if I try to rebuild libjulia 1.5, won't I get a version that is to new?

[fingolfin]

For most dependencies, it doesn't matter which version we get, as they do not affect the Julia ABI, and that's all that counts for this JLL.

(That said, I wonder if not more of those deps should be turned into BuildDeps and/or be dropped because Julia now actually will fetch the relevant binaries anyway -- we only need to keep things that are needed to link against libjulia after all (i.e. dependencies of the shared library) and/or needed to include headers. But e.g. I don't see why LibGit2_jll would need to be a dependency.

Looking at libjulia.dylib and libjulia-internal.dylib on macOS, I see this:

$ otool -L usr/lib/libjulia.dylib
usr/lib/libjulia.dylib:
	@rpath/libjulia.dylib (compatibility version 1.0.0, current version 1.7.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
$ otool -L usr/lib/libjulia-internal.dylib
usr/lib/libjulia-internal.dylib:
	@rpath/libjulia-internal.dylib (compatibility version 1.0.0, current version 1.7.0)
	@rpath/libjulia.dylib (compatibility version 1.0.0, current version 1.7.0)
	@rpath/libunwind.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	@rpath/libLLVM.dylib (compatibility version 1.0.0, current version 12.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1575.17.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

Based on that, I wonder if we can't turn all dependencies into BuildDependency with the following exeptions:

• libLLVM_jll resp. LLVM_full_jll
• LibUnwind_jll resp. LibOSXUnwind_jll resp. LLVMLibUnwind_jll

But I am not 100% sure, perhaps I am missing something? I think to be sure, one also needs to experiment and see if a JLL build this way then can be used to e.g. build libcxxwrap_julia?

[vchuravy]

I think that would actually be a step in the right direction. For libjulia we only want to define the dependencies the runtime itself uses and not the standard lib etc.

[fingolfin]

Thanks @giordano I've now enabled the "missing" ARM platforms and they all build! I've also changed most Deps into BuildDeps.