Bug: DFTK fails to precompile on a cluster

[m-schwendt]

Precompilation of DFTK fails on a cluster, with the following error message:

PLEASE submit a bug report to https://github.com/julialang/julia and include the crash backtrace.
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  libLLVM-16jl.so 0x000015240492bd38 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 40
1  libLLVM-16jl.so 0x000015240492985c
2  libpthread.so.0 0x0000152409af9cf0
3  libc.so.6       0x00001524035bb144
4  lld             0x00000000005df633 void lld::elf::InputSection::writeTo<llvm::object::ELFType<(llvm::support::endianness)1, true>>(unsigned char*) + 179
5  lld             0x00000000006055a0
6  libLLVM-16jl.so 0x000015240487fa94
7  libLLVM-16jl.so 0x00001524048813d9
8  libstdc++.so.6  0x0000152403d17b23
9  libpthread.so.0 0x0000152409aef1ca
10 libc.so.6       0x0000152403525e73 clone + 67
  ◑ DFTK

The stacktrace is:

 top-level scope at REPL[17]:1
 eval at boot.jl:430 [inlined]
 eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module) at REPL.jl:261
 repl_backend_loop(backend::REPL.REPLBackend, get_module::Function) at REPL.jl:368
 start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function) at REPL.jl:343
 kwcall(::NamedTuple, ::typeof(REPL.start_repl_backend), backend::REPL.REPLBackend, consumer::Any) at REPL.jl:340
 run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any) at REPL.jl:500
 run_repl(repl::REPL.AbstractREPL, consumer::Any) at REPL.jl:486
 (::Base.var"#1150#1152"{Bool, Symbol, Bool})(REPL::Module) at client.jl:446
 #invokelatest#2 at essentials.jl:1055 [inlined]
 invokelatest at essentials.jl:1052 [inlined]
 run_main_repl(interactive::Bool, quiet::Bool, banner::Symbol, history_file::Bool, color_set::Bool) at client.jl:430
 repl_main at client.jl:567 [inlined]
 _start() at client.jl:541

I'm using Julia 1.11.6. As far as I can see the issue only occurs with the DFTK package, but since the error specifically says to create the issue on JuliaLang, I'm posting it here.

[Seelengrab]

Hi! How did you install Julia?

[giordano]

Set JULIA_NUM_PRECOMPILE_TASKS to a low value (enough not to spawn too many processes violating whatever restriction there's on your system).

[m-schwendt]

Thanks for the answers! I got it to run again.

Hi! How did you install Julia?

I used juliaup as recommended

Set JULIA_NUM_PRECOMPILE_TASKS to a low value (enough not to spawn too many processes violating whatever restriction there's on your system).

Thanks for the suggestion. I also had to delete all the packages in the .julia/compiled/v1.11 folder, but then it worked (I probably should have done that before posting this issue and setting the environment variable). Anyway, if possible, it would be nice if the error message could contain some additional information.

[giordano]

if possible, it would be nice if the error message could contain some additional information.

"if possible" is doing lots of work here. It's not obvious to programmatically know why the precompilation job was terminated, but if you say it's on a login node of a cluster then I can try and guess it. As I already said in the linked issue, it's also not possible to query in general what are the restrictions set on the system, as there are many different ways to do that, and many of them are not querable from normal users. And when restrictions are posed on memory usage, we can't guess in advance how much memory a precompilation task is going to use.