cannot import nonprecompiled packages from embedding interface

[mlubin]

If I have a call, e.g.,

jl_eval_string("import MathProgBase");

and the MathProgBase package hasn't been precompiled, then I get an exception:

Base.UVError(prefix="could not spawn `/mnt/hdd/mlubin/julia-0.4/julia-bb73f3489d/lib/julia -Cnative -J/mnt/hdd/mlubin/julia-0.4/julia-bb73f3489d/lib/julia/sys.so --output-ji /home/mlubin/.julia/lib/v0.4/MathProgBase.ji --output-incremental=yes --startup-file=no --history-file=no --color=no --eval 'while !eof(STDIN)
    eval(Main, deserialize(STDIN))
end

The path /mnt/hdd/mlubin/julia-0.4/julia-bb73f3489d/lib/julia is a directory, not the julia binary.

[vtjnash]

you've probably set JULIA_HOME wrong. it should point to the julia executable.

[mlubin]

I haven't set JULIA_HOME. I call

jl_init(JULIA_INIT_DIR);

with -DJULIA_INIT_DIR=\"/mnt/hdd/mlubin/julia-0.4/julia-bb73f3489d/lib\" where that define is produced by julia-config.jl --cflags.

[KristofferC]

Ref: https://groups.google.com/forum/?nomobile=true#!topic/julia-users/2sR6QdPLA80

I do the same as @mlubin. In CMakeLists.txt:

# Julia stuff
set(JULIA_PATH /home/kristoffer/julia0.4)
set(JULIA_LIB_DIR ${JULIA_PATH}/usr/lib)
add_definitions(-DJULIA_INIT_DIR="${JULIA_LIB_DIR}")

and in c++:

  jl_init(JULIA_INIT_DIR);

[mlubin]

@KristofferC, is it working or broken for you with that configuration?

[KristofferC]

Broken as in the julia user post.

[mlubin]

Bump, seems like a potential blocker for 0.5 to not be able to safely import packages from the embedding interface.
CC @SteveDiamond

[ihnorton]

Worked on this with @KristofferC and reproduced using ForwardDiff. The issue was that jl_init (or JULIA_HOME) needs to be the path containing the julia executable -- not the lib path, and not the path to the executable itself. Please re-open if needed.

[KristofferC]

Shouldn't the julia-config.jl script be modified to not point to the lib folder?

[Keno]

Yes

[Keno]

Also, why is this needed at all? Is the path from libjulia to the system image not fixed?

[ihnorton]

jl_init(NULL) works fine too. I'm not sure what the history is.

[ihnorton]

As far as I can tell there is nothing to be done here. There have been a variety of changes to init.c, and the test script now works with default --cflags output, and still works with jl_init(NULL).

[ihnorton]

Changing the embedding example code to:

int main()
{
    jl_init();

    jl_eval_string("import MathProgBase");
    if (jl_exception_occurred()) {
        jl_call2(jl_get_function(jl_base_module, "show"), jl_stderr_obj(), jl_exception_occurred());
        jl_printf(jl_stderr_stream(), "\n");
    }
}

gives

could not spawn `/Users/inorton/git/julia/usr/lib/julia -Cnative -J/Users/inorton/git/julia/usr/lib/julia/sys.dylib --compile=yes --depwarn=yes -O0 --output-ji /Users/inorton/.julia/lib/v0.7/MathProgBase.ji --output-incremental=yes --startup-file=no --history-file=no --color=no --eval 'while !eof(STDIN)
    eval(Main, deserialize(STDIN))
end
'`: permission denied (EACCES)

So I guess this is still/again an issue.

Ref: https://discourse.julialang.org/t/embedding-jullia-jl-init-julia-init-and-julia-home/4032

[Non-Contradiction]

I ran into the same issue in developing JuliaCall which embeds julia in R. I currently walk around the problem by doing things like system("julia -e 'using MathProgBase') in R first, which compiles the package, then I can just import the package in embedded julia.