Julia precompiling error on cluster

[henrikjaerleblad]

Hi,

I am trying to precompile my Julia packages on our university cluster, when I get the following error:

When instantiating my project, I got this error (prior to pre-compiling):

We have a policy on the cluster, that no-one is allowed to use to much resources without submitting a job to the SLURM workload manager. Otherwise, the process will be automatically killed. I am suspecting that the system shuts of Julia's access to resources (multiple CPUs, RAM etc) when Julia is attempting to pre-compile as fast as possible. But the earlier warning message might indicate something else.

Does anyone have any idea on how to solve this?

[Endilll]

Try recompiling with LLVM_PARALLEL_COMPILE_JOBS=1 and LLVM_PARALLEL_LINK_JOBS=1 (documented here). There is no enough information for us about the crash, and it could be caused by your system shutting Clang down anyway.

[giordano]

I think you want to set the environment variable JULIA_NUM_PRECOMPILE_TASKS=1. The cmake options suggested above are for building llvm, which has nothing to do with the reported issue.

[nhz2]

I had a similar problem, and running ulimit -u 10000 before starting Julia fixed the issue.

[henrikjaerleblad]

The suggestion from @giordano worked. The SLURM workload manager would automatically shut off Julia's access to resources, as soon as it tried to use more than one CPU core (without having submitted pre-compilation as a job).

Thanks a lot guys!

[henrikjaerleblad]

I will now close the issue.