Error when `using SDPAFamily`

[ericphanson]

Sometimes I get this error when calling using SDPAFamily:

ERROR: could not load library "/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia/sys.dylib"
dlopen(/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia/sys.dylib, 6): Symbol not found: _jl_apply_2va
  Referenced from: /Applications/Julia-1.0.app/Contents/Resources/julia/lib/julia/sys.dylib
  Expected in: /Applications/Julia-1.3.app/Contents/Resources/julia/bin/../lib/libjulia.dylib
 in /Applications/Julia-1.0.app/Contents/Resources/julia/lib/julia/sys.dylib

I think it comes from using SDPAFamily from different Julia versions, and needing to rebuild the package. I'm not quite sure how to fix it; I think one easy fix might just be to remove the plain-SDPA functionality and only support the high-precision variants, since then we don't need to link against anything.

[ericphanson]

This is because of eg

    libpaths = split(get(ENV, "DYLD_LIBRARY_PATH", ""), ":")
    if !("/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia" in libpaths)
        push!(libpaths, "/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia")
    end

being present in the deps.jl file generated by BinaryProvider. Not sure why it's doing Julia-version-specific things, but it can be worked around by re-building the package when switching Julia versions. My full deps.jl file is the following:

## This file autogenerated by BinaryProvider.write_deps_file().
## Do not edit.
##
## Include this file within your main top-level source, and call
## `check_deps()` from within your module's `__init__()` method

if isdefined((@static VERSION < v"0.7.0-DEV.484" ? current_module() : @__MODULE__), :Compat)
    import Compat.Libdl
elseif VERSION >= v"0.7.0-DEV.3382"
    import Libdl
end
const sdpa_gmp = joinpath(dirname(@__FILE__), "usr/bin/sdpa_gmp")
const libqd = joinpath(dirname(@__FILE__), "usr/lib/libqd.0.dylib")
const sdpa_qd = joinpath(dirname(@__FILE__), "usr/bin/sdpa_qd")
const libqd = joinpath(dirname(@__FILE__), "usr/lib/libqd.0.dylib")
const sdpa_dd = joinpath(dirname(@__FILE__), "usr/bin/sdpa_dd")
const libqd = joinpath(dirname(@__FILE__), "usr/lib/libqd.0.dylib")
const sdpa = joinpath(dirname(@__FILE__), "usr/bin/sdpa")
const libsdpa = joinpath(dirname(@__FILE__), "usr/lib/libsdpa.0.dylib")
function check_deps()
    global sdpa_gmp
    if !isfile(sdpa_gmp)
        error("$(sdpa_gmp) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    global libqd
    if !isfile(libqd)
        error("$(libqd) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    if Libdl.dlopen_e(libqd) in (C_NULL, nothing)
        error("$(libqd) cannot be opened, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    global sdpa_qd
    if !isfile(sdpa_qd)
        error("$(sdpa_qd) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    global libqd
    if !isfile(libqd)
        error("$(libqd) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    if Libdl.dlopen_e(libqd) in (C_NULL, nothing)
        error("$(libqd) cannot be opened, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    global sdpa_dd
    if !isfile(sdpa_dd)
        error("$(sdpa_dd) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    global libqd
    if !isfile(libqd)
        error("$(libqd) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    if Libdl.dlopen_e(libqd) in (C_NULL, nothing)
        error("$(libqd) cannot be opened, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    global sdpa
    if !isfile(sdpa)
        error("$(sdpa) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    global libsdpa
    if !isfile(libsdpa)
        error("$(libsdpa) does not exist, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    if Libdl.dlopen_e(libsdpa) in (C_NULL, nothing)
        error("$(libsdpa) cannot be opened, Please re-run Pkg.build(\"SDPAFamily\"), and restart Julia.")
    end

    libpaths = split(get(ENV, "DYLD_LIBRARY_PATH", ""), ":")
    if !("/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia" in libpaths)
        push!(libpaths, "/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia")
    end
    ENV["DYLD_LIBRARY_PATH"] = join(filter(!isempty, libpaths), ":")

end

const HAS_WSL = Dict{Symbol,Bool}(Dict(:sdpa_dd => 0,:sdpa_gmp => 0,:sdpa_qd => 0,:sdpa => 0))

[ericphanson]

I think this will be solved by switching SDPA to a JLL package when JuliaPackaging/Yggdrasil#804 is finished.

[ViralBShah]

Now in JuliaPackaging/Yggdrasil#1478. It would be great if folks from here who may know the cxxwrap related internals better can chime in on that PR. I believe we should be fairly close.

[ericphanson]

That’s awesome! I have to say, I really don’t know the Cxxwrap internals at all, I just copied them from https://github.com/jump-dev/SDPA.jl (I think @blegat has done a lot of work for that package, so he might know better)

[ViralBShah]

We definitely need updates on the cxxwrap. See comments in the Yggdrasil PR.

[ViralBShah]

This is merged now. Can we try it out? Do see the comments in the Yggdrasil PR about the gfortran issue- but it should hopefully just work.

[ericphanson]

just to follow up, hopefully the SDPA JLL package can help here once the gfortran issues are sorted out (I can't really help at all, all this build stuff is very far from my experience).

For SDPAFamily to be fully JLL'd, we'd need the high-precision variants SDPA-GMP, SDPA-DD, and SDPA-QD as well to be built with the new binarybuilder. (Note https://github.com/jump-dev/SDPA.jl does not support these and just uses SDPA itself). They are a bit strange in that they don't expose a programmatic API, but just a binary executable to shell out to (and in fact, we just write a file and ask it to read the file). I tried to build QD in JuliaPackaging/Yggdrasil#380, one of the dependencies of these binaries, but there is something wrong with the build: JuliaPackaging/Yggdrasil#755. We actually don't need to load any libraries into the Julia process, however, since we only shell out to the final binaries, so this might not be an issue for that use-case.

I think the new scratch spaces (which is now in https://github.com/JuliaPackaging/Scratch.jl I believe) can help make the old BinaryBuilder/BinaryProvider setup work in a better way, however. We can just make a scratch space to keep the deps files instead of mutating the package directory. We could even do that for the old way of building SDPA (also keying the scratch space off the Julia version) which would eliminate the error in this issue without needing the JLL.

Separately, we can also use the scratch spaces to store the files we read and write to interact with the binaries; right now we use mktempdir() but I got one email suggesting a user was having permissions issues with this (I suggested using the environmental variable TMPDIR to customize the location), and it might be better to use a scratch space instead.