Precompilation error on 1.6 with macOS 10.13

[giacomogiudice]

Hi,

I ran into this issue while trying to add Plots. Building GR seems to go fine and at precompilation it errors.

(@v1.6) pkg> add GR
  Installing known registries into `~/.julia`
[...]
   Building GR → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/f74b42150042d11a1d94badf27c6125867c7e9bd/build.log`
Precompiling project...
  Progress [========================================>]  74/74
  ✗ GR_jll
  ✗ GR
72 dependencies successfully precompiled in 72 seconds
2 dependencies errored

Attempting to precompile manually seems to throw on error about missing Qt5 libraries

julia> import Pkg

julia> Pkg.precompile()
Precompiling project...
  Progress [========================================>]  2/2
  ✗ GR_jll
  ✗ GR
0 dependencies successfully precompiled in 7 seconds (72 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

GR [28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71]

ERROR: LoadError: LoadError: InitError: could not load library "/Users/giacomo/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtConcurrent.framework/Versions/5/QtConcurrent"
dlopen(/Users/giacomo/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtConcurrent.framework/Versions/5/QtConcurrent.dylib, 1): image not found
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:114
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:114
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/WnWcZ/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ Qt5Base_jll ~/.julia/packages/Qt5Base_jll/Ks8eF/src/wrappers/x86_64-apple-darwin.jl:31
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:674
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:760
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:998
  [8] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:914
  [9] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:901
 [10] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
 [11] top-level scope
    @ ~/.julia/packages/JLLWrappers/WnWcZ/src/toplevel_generators.jl:170
 [12] include
    @ ./Base.jl:386 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1213
 [14] top-level scope
    @ none:1
 [15] eval
    @ ./boot.jl:360 [inlined]
 [16] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [17] top-level scope
    @ none:1
during initialization of module Qt5Base_jll
in expression starting at /Users/giacomo/.julia/packages/GR_jll/qVgdC/src/wrappers/x86_64-apple-darwin.jl:13
in expression starting at /Users/giacomo/.julia/packages/GR_jll/qVgdC/src/GR_jll.jl:2

and yet the supposedly missing library seems to be there

~ ❯❯❯ la /Users/giacomo/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtConcurrent.framework/Versions/5/
total 104
drwxr-xr-x  24 giacomo  staff   768B Apr  7 11:19 Headers
-r-xr-xr-x   1 giacomo  staff    48K Apr  7 11:19 QtConcurrent
drwxr-xr-x   4 giacomo  staff   128B Apr  7 11:19 Resources

Here is the versioninfo

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, sandybridge)

(@v1.6) pkg> st
      Status `~/.julia/environments/v1.6/Project.toml`
  [28b8d3ca] GR v0.57.1

[jheinen]

Could you please try the "old" setup:

ENV["JULIA_GR_PROVIDER"]="GR"
] build GR

I just want to make sure, that there is no problem with your macOS system environment.

[giacomogiudice]

Still builds fine

julia> ENV["JULIA_GR_PROVIDER"]="GR"
"GR"

julia> ENV["JULIA_GR_PROVIDER"]
"GR"

(@v1.6) pkg> build GR
    Building GR → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/f74b42150042d11a1d94badf27c6125867c7e9bd/build.log`

but precompilation still fails

[jheinen]

but precompilation still fails

What is the error message (or the content of build.log)?

[jheinen]

Our own build system is 10.13. So it should work with "GR" as binary provider. You probably have to remove GR (and GR_jll) ans retry.

[giacomogiudice]

I wiped the whole ~/.julia/ directory before installing.
build.log is just

[ Info: Downloading pre-compiled GR 0.57.1 Darwin binary

An excerpt of the error message while precompiling is in the first message.

[jheinen]

If this is the error, the GR still uses GR_jll:

ERROR: LoadError: LoadError: InitError: could not load library "/Users/giacomo/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtConcurrent.framework/Versions/5/QtConcurrent"
dlopen(/Users/giacomo/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtConcurrent.framework/Versions/5/QtConcurrent.dylib, 1): image not found
Stacktrace:```

I will check it ...

[giacomogiudice]

Thanks for your quick answers. The full stacktrace is

julia> Pkg.precompile()
Precompiling project...
  Progress [========================================>]  2/2
  ✗ GR_jll
  ✗ GR
0 dependencies successfully precompiled in 15 seconds (72 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

GR [28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71]

ERROR: LoadError: LoadError: InitError: could not load library "/Users/giacomo/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtConcurrent.framework/Versions/5/QtConcurrent"
dlopen(/Users/giacomo/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtConcurrent.framework/Versions/5/QtConcurrent.dylib, 1): image not found
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:114
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:114
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/WnWcZ/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ Qt5Base_jll ~/.julia/packages/Qt5Base_jll/Ks8eF/src/wrappers/x86_64-apple-darwin.jl:31
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:674
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:760
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:998
  [8] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:914
  [9] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:901
 [10] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
 [11] top-level scope
    @ ~/.julia/packages/JLLWrappers/WnWcZ/src/toplevel_generators.jl:170
 [12] include
    @ ./Base.jl:386 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1213
 [14] top-level scope
    @ none:1
 [15] eval
    @ ./boot.jl:360 [inlined]
 [16] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [17] top-level scope
    @ none:1
during initialization of module Qt5Base_jll
in expression starting at /Users/giacomo/.julia/packages/GR_jll/qVgdC/src/wrappers/x86_64-apple-darwin.jl:13
in expression starting at /Users/giacomo/.julia/packages/GR_jll/qVgdC/src/GR_jll.jl:2
ERROR: LoadError: LoadError: Failed to precompile GR_jll [d2c73de3-f751-5644-a686-071e5b155ba9] to /Users/giacomo/.julia/compiled/v1.6/GR_jll/jl_hxV4sh.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IOContext{Base.PipeEndpoint}, internal_stdout::IOContext{IOStream})
    @ Base ./loading.jl:1360
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1306
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1021
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:914
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:901
  [7] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [8] include(x::String)
    @ GR ~/.julia/packages/GR/f111O/src/GR.jl:3
  [9] top-level scope
    @ ~/.julia/packages/GR/f111O/src/GR.jl:15
 [10] include
    @ ./Base.jl:386 [inlined]
 [11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1213
 [12] top-level scope
    @ none:1
 [13] eval
    @ ./boot.jl:360 [inlined]
 [14] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [15] top-level scope
    @ none:1
in expression starting at /Users/giacomo/.julia/packages/GR/f111O/deps/deps.jl:1
in expression starting at /Users/giacomo/.julia/packages/GR/f111O/src/GR.jl:3
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:55
 [2] precompile(ctx::Pkg.Types.Context; internal_call::Bool, strict::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:1225
 [3] precompile
   @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:914 [inlined]
 [4] #precompile#194
   @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:912 [inlined]
 [5] precompile()
   @ Pkg.API /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:912
 [6] top-level scope
   @ REPL[4]:1

[jheinen]

That's what I get:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0 (2021-03-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.6) pkg> st
      Status `~/.julia/environments/v1.6/Project.toml` (empty project)

julia> ENV["JULIA_GR_PROVIDER"]="GR"
"GR"

(@v1.6) pkg> add GR
  Installing known registries into `~/.julia`
       Added registry `General` to `~/.julia/registries/General`
   Resolving package versions...
...
    Building GR → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/f74b42150042d11a1d94badf27c6125867c7e9bd/build.log`
  Progress [========================================>]  74/74
  ✗ GR_jll
73 dependencies successfully precompiled in 44 seconds
1 dependency errored

julia> using GR

julia> ENV["GRDIR"]
"/Users/softadm/.julia/packages/GR/f111O/src/../deps/gr"

julia> plot(randn(10))

julia> version()
"0.57.1"

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, sandybridge)
Environment:
  JULIA_GR_PROVIDER = GR

As you can see, building GR_jll on an old macOS System fails. But as I changed to the old provider (before adding GR), it still works fine, because the binaries (that we built) are used.

Then I tried the same procedure on a macOS 11.2.3 and everything worked, too:

    Building GR → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/f74b42150042d11a1d94badf27c6125867c7e9bd/build.log`
  Progress [========================================>]  74/74
74 dependencies successfully precompiled in 16 seconds

So, the problem is in the GR_jll package.

We use the original Qt5 distribution for our binaries. Therefore I suspect a problem in the BinaryBuilder Qt5 package. I can't help at this point - we already support a dozen different distribution paths and languages. If I now get deep into BinaryBuilder as well, I won't have time to develop things further. I ask for your understanding.

So as a solution I can only suggest the use of our binaries (using the JULIA_GR_PROVIDER method) or a macOS update of you computer.

BinaryBuilder is a great system, but I had concerns from the beginning and it shows again that the deployment of software libraries is underestimated.

[giacomogiudice]

I obtain the same with JULIA_GR_PROVIDER. Thanks for taking time to look into this.

So as a solution I can only suggest the use of our binaries (using the JULIA_GR_PROVIDER method) or a macOS update of you computer.

The machine is a bit old, and apple doesn't officially support updates. Although try third-party patches do exist...

I'll go ahead and close the issue for now.