LoadError: InitError during Package Build

[pedromxavier]

Thanks again for this package!

I'm currently trying to build my recently released package Anneal.jl and this error is showing up during precompilation. One must run ] precompile to see the message below:

ERROR: LoadError: InitError: failed process: Process(`'C:\Users\pedroxavier\.julia\artifacts\d628738c351341856bc6e8abe94cd6c8d8a7bf66\Library\bin\micromamba.exe' -r 'C:\Users\pedroxavier\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root' create -y -p 'C:\Users\pedroxavier\.julia\environments\v1.7\.CondaPkg\env' --override-channels --no-channel-priority pip 'python >=3.5,<4' 'python >=3.7,<4' -c anaconda -c conda-forge`, ProcessExited(3221226505)) [3221226505]

A way to fix this is to plug the command between backticks in a Julia run call.

run(`'C:\Users\pedroxavier\.julia\artifacts\d628738c351341856bc6e8abe94cd6c8d8a7bf66\Library\bin\micromamba.exe' -r 'C:\Users\pedroxavier\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root' create -y -p 'C:\Users\pedroxavier\.julia\environments\v1.7\.CondaPkg\env' --override-channels --no-channel-priority pip 'python >=3.5,<4' 'python >=3.7,<4' -c anaconda -c conda-forge`)

This way, micro mamba gets installed and everything works fine. The problem is that my package won't precompile directly without this trick. I guess it has something to do with privileges, chmod or something like that.

To reproduce this issue, it is enough to run

] add Anneal
] precompile

Anneal is available in the General Registry, so the first line will error after a little while and ] precompile will reveal the message above.

[cjdoris]

How strange. I can reproduce this on your package, and on a blank package also using PythonCall.

There are a few ways forward:

1. Figure out why the micromamba call is failing. I have run it again with really high verbosity, and the output looks normal until it stops, no error messages or anything. Looks like it stops just before it tries to download the conda repodata. It's not clear if this is even a problem with CondaPkg or Julia or MicroMamba.
2. If it is possible to detect that a package is being loaded just to be precompiled, I could add a check for this into PythonCall and have it skip most of its initialisation.
3. Modify PythonCall to not do any initialisation until you actually do something with it. This is something I've mused about for a while, and is doable, and would work around this issue.

Options 2 and 3 are really just workarounds, but might be nice to have anyway. Option 1 is the real solution, but I'm not sure how to approach it, I may need to ask for help on discourse.

[pedromxavier]

Lazy installation as in 3 seems to be the easiest path to follow, but also one of the most painful issues I've dealt with while trying to use PyCall.

I've runned as admin and the problem is still there.

There is one more weird fact: CI's are running just fine!

[cjdoris]

You don't need to run the micromamba command yourself - even if ] precompile fails you can still do using Anneal and it will precompile successfully. I think the issue is isolated to precompiling packages ahead of time, as opposed to precompiling them just-in-time when you actually use them. This probably explains why CI works OK - it just doesn't precompile ahead of time.

Option 3 is certainly doable, but will be the most disruptive and I don't want to rush into it. Options 1 and 2 solve the problem without changing any outward behaviour so I'd rather do those first if possible.

[cjdoris]

BTW this only affects Windows - on Mac and Linux it precompiles fine. Though it has pointed out an undesirable effect, which is that CondaPkg resolves all its dependencies while precompiling, which hides its logs.

[pedromxavier]

Hello again, @cjdoris!

I've noticed that some of PythonCall's dependecies had new releases and tried to build my packages again, having no trouble this time 😄. Do you think this issue is solved for good?

[cjdoris]

No idea but I'm glad it's working! I didn't do anything specifically to fix this. What updated?

[pedromxavier]

Hmmm then it must have something to do with some kind of caching 🤔 I installed them in many environments within v1.7 and also on another installation of v1.6 (both on windows) and no problems occured

[cjdoris]

The precompiled module will be cached in ~/.julia/compiled - deleting the right directory in there (or the whole directory) should reproduce the issue.

[cjdoris]

Unless it's magically fixed. I possibly updated the version of MicroMamba bundled with MicroMamba.jl since you filed this issue, that may have solved it.

[YuliyaCl]

Hello!
I have the same error during adding PythonCall to the project - Precompiling of PythonCall failed with the same error as in the issue.
I've tried to fix it using pedromxavier's recommendation but the error still exist.

The full error text:

Failed to precompile testWFDB [c64484d1-08e2-4c4d-8ee9-fe33f95c1a54] to C:\Users\yzh\.julia\compiled\v1.7\testWFDB\jl_1FAC.tmp.
    CondaPkg Found dependencies: C:\Users\yzh\.julia\packages\PythonCall\XgP8G\CondaPkg.toml
    CondaPkg Resolving changes
             + python
    CondaPkg Installing packages
warning  libmamba Cannot lock 'C:\Users\yzh\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root\pkgs\cache'
    Waiting for other mamba process to finish
error    libmamba Path already locked by the same PID: 'C:\Users\yzh\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root\pkgs\cache'
warning  libmamba LockFile creation for path 'C:\Users\yzh\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root\pkgs\cache' failed, continuing without it
error    libmamba Path already locked by the same PID: 'C:\Users\yzh\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root\pkgs\cache'
ERROR: LoadError: InitError: failed process: Process(`'C:\Users\yzh\.julia\artifacts\d628738c351341856bc6e8abe94cd6c8d8a7bf66\Library\bin\micromamba.exe' -r 'C:\Users\yzh\.julia\scratchspaces\0b3b1443-0f03-428d-bdfb-f27f9c1191ea\root' create -q -y -p 'c:\Users\yzh\.julia\dev\testWFDB\.CondaPkg\env' --override-channels --no-channel-priority "python[version='>=3.5,<4']" -c conda-forge`, ProcessExited(3221226505)) [3221226505]

Stacktrace:
  [1] pipeline_error
    @ .\process.jl:531 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base .\process.jl:446
  [3] run(::Cmd)
    @ Base .\process.jl:444
  [4] _run(io::IO, cmd::Cmd, args::Any; flags::Any)
    @ CondaPkg C:\Users\yzh\.julia\packages\CondaPkg\Olo5Y\src\resolve.jl:283
  [5] _resolve_conda_install(io::Any, conda_env::Any, specs::Any, channels::Any; create::Any)
    @ CondaPkg C:\Users\yzh\.julia\packages\CondaPkg\Olo5Y\src\resolve.jl:191
  [6] resolve(; force::Bool, io::IO, interactive::Bool, dry_run::Bool)
    @ CondaPkg C:\Users\yzh\.julia\packages\CondaPkg\Olo5Y\src\resolve.jl:395
  [7] resolve()
    @ CondaPkg C:\Users\yzh\.julia\packages\CondaPkg\Olo5Y\src\resolve.jl:288
  [8] envdir()
    @ CondaPkg C:\Users\yzh\.julia\packages\CondaPkg\Olo5Y\src\env.jl:70
  [9] init_context()
    @ PythonCall.C C:\Users\yzh\.julia\packages\PythonCall\XgP8G\src\cpython\context.jl:56
 [10] __init__()
    @ PythonCall.C C:\Users\yzh\.julia\packages\PythonCall\XgP8G\src\cpython\CPython.jl:21
 [11] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base .\loading.jl:768
 [12] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base .\loading.jl:854
 [13] _require(pkg::Base.PkgId)
    @ Base .\loading.jl:1097
 [14] require(uuidkey::Base.PkgId)
    @ Base .\loading.jl:1013
 [15] require(into::Module, mod::Symbol)
    @ Base .\loading.jl:997
 [16] include
    @ .\Base.jl:418 [inlined]
 [17] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base .\loading.jl:1318
 [18] top-level scope
    @ none:1
 [19] eval
    @ .\boot.jl:373 [inlined]
 [20] eval(x::Expr)
    @ Base.MainInclude .\client.jl:453
 [21] top-level scope
    @ none:1
during initialization of module C
in expression starting at c:\Users\yzh\.julia\dev\testWFDB\src\testWFDB.jl:1
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types C:\Users\yzh\AppData\Local\Programs\Julia-1.7.1\share\julia\stdlib\v1.7\Pkg\src\Types.jl:68
 [2] precompile(ctx::Pkg.Types.Context; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API C:\Users\yzh\AppData\Local\Programs\Julia-1.7.1\share\julia\stdlib\v1.7\Pkg\src\API.jl:1362
 [3] precompile
   @ C:\Users\yzh\AppData\Local\Programs\Julia-1.7.1\share\julia\stdlib\v1.7\Pkg\src\API.jl:1013 [inlined]
 [4] #precompile#220
   @ C:\Users\yzh\AppData\Local\Programs\Julia-1.7.1\share\julia\stdlib\v1.7\Pkg\src\API.jl:1011 [inlined]
 [5] precompile()
   @ Pkg.API C:\Users\yzh\AppData\Local\Programs\Julia-1.7.1\share\julia\stdlib\v1.7\Pkg\src\API.jl:1011
 [6] top-level scope
   @ REPL[10]:1

Do you have any ideas what I need to check...? I've used PyCall.jl in another project, maybe some conflicts occured.

Thanks.

UPD 11/04/22: deleting of cache folders inside ....\.julia\scratchspaces solved the problem.

[cjdoris]

I can no longer reproduce this issue. Anyone else got a way to reproduce it?

[pedromxavier]

@cjdoris me neither. Since my last comment everything seems to work just fine. It seems to be something that was fixed in a dependency.

I suggest you to create a release limiting PythonCall's dependencies with upper bounds, to assure that this keeps working. Would you like me to open another issue to address it?

[josephsdavid]

I am having the same issue on linux, I have one package which wraps the python package wandb, which I am loading in a completely separate package. When I do so, I was having a hard time, getting init errors saying wandb was missing. To resolve these, two approaches worked:

1. in the repl, ]instantiate, ]build, ]precompile (I am not sure which one exactly worked), then import the package
2. on the new package which depends on the wrapper package, add condapkg as a direct dependency, and then everything works for some reason

[josephsdavid]

I think I have figured out at least a big clue as to what is going on. If I remove the .CondaPkg folder before i try precompiling, everything works as it should!

[cjdoris]

@josephsdavid Do you have a way to reproduce this?

[josephsdavid]

@josephsdavid Do you have a way to reproduce this?

Let me try messing around with it this evening (12 ish hours from now!)

[josephsdavid]

So currently having a hard time reproducing, I will need to spend a bit more time figuring out how the situation which caused my issue arose

[josephsdavid]

So the conditions for the problem arising still remain difficult to reproduce, but with N=3 can confirm that nuking .CondaPkg/ solves the trick

[pedromxavier]

I've just runned into this issue one more time and I guess it has something to do with mixing linux and windows filesystems. Switching back from wsl to windows solved it locally by now. @josephsdavid where are you running your scenario? Trashing .CondaPkg/ wasn't enough to solve the issue in wsl.

[josephsdavid]

I've just runned into this issue one more time and I guess it has something to do with mixing linux and windows filesystems. Switching back from wsl to windows solved it locally by now. @josephsdavid where are you running your scenario? Trashing .CondaPkg/ wasn't enough to solve the issue in wsl.

for me it is perhaps a separate issue, happened both on macOS and Linux

[cjdoris]

Important note: I can't do anything to help without more specific information. So please, the next time anyone encounters this issue, can you report:

• the full error message
• what you were doing that led to it (e.g. include your command line history)
• the contents of .CondaPkg (ideally save off .CondaPkg so we can investigate more later)
• versioninfo()
• Pkg.status()
• CondaPkg.status()
• anything else relevant that you can think of

The more detail the better! Thank you.

[pedromxavier]

The specific building problem has changet a lot since my first message. The original pitfall is mostly solved I guess, since I've not been able to reproduce since v0.8 came. I'm closing this one for good and submitting a new one sticking to the info you asked for. Thanks!