Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System-wide CUDA in LD_LIBRARY_PATH breaks CUBLAS #1755

Closed
jfeist opened this issue Feb 4, 2023 · 10 comments
Closed

System-wide CUDA in LD_LIBRARY_PATH breaks CUBLAS #1755

jfeist opened this issue Feb 4, 2023 · 10 comments
Labels
bug Something isn't working

Comments

@jfeist
Copy link

jfeist commented Feb 4, 2023

Describe the bug

The norm(x) function seems to be broken in CUDA.jl v4.

To reproduce

The Minimal Working Example (MWE) for this bug:

using CUDA, LinearAlgebra
x = CUDA.fill(1., 1)
norm(x)

gives the error:

ERROR: LoadError: DivideError: integer division error
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/libcublas.jl:106 [inlined]
 [2] macro expansion
   @ ~/.julia/packages/CUDA/ZX8mg/src/pool.jl:312 [inlined]
 [3] macro expansion
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/libcublas.jl:22 [inlined]
 [4] cublasDnrm2_v2(handle::Ptr{CUDA.CUBLAS.cublasContext}, n::Int64, x::CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}, incx::Int64, result::Base.RefValue{Float64})
   @ CUDA.CUBLAS ~/.julia/packages/CUDA/ZX8mg/lib/utils/call.jl:26
 [5] nrm2
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/wrappers.jl:168 [inlined]
 [6] nrm2
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/wrappers.jl:173 [inlined]
 [7] norm
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/linalg.jl:108 [inlined]
 [8] norm(x::CuArray{Float64, 1, CUDA.Mem.DeviceBuffer})
   @ CUDA.CUBLAS ~/.julia/packages/CUDA/ZX8mg/lib/cublas/linalg.jl:107
 [9] top-level scope
   @ ~/cudabug/tmp.jl:4
in expression starting at ~/cudabug/tmp.jl:4
Manifest.toml

# This file is machine-generated - editing it directly is not advised

julia_version = "1.9.0-beta3"
manifest_format = "2.0"
project_hash = "4d6f52122ea9741175c8c71d00021be0a921f3ad"

[[deps.AbstractFFTs]]
deps = ["ChainRulesCore", "LinearAlgebra"]
git-tree-sha1 = "69f7020bd72f069c219b5e8c236c1fa90d2cb409"
uuid = "621f4979-c628-5d54-868e-fcf4e3e8185c"
version = "1.2.1"

[[deps.Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "0310e08cb19f5da31d08341c6120c047598f5b9c"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.5.0"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
version = "1.1.1"

[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[deps.BFloat16s]]
deps = ["LinearAlgebra", "Printf", "Random", "Test"]
git-tree-sha1 = "dbf84058d0a8cbbadee18d25cf606934b22d7c66"
uuid = "ab4f0b2a-ad5b-11e8-123f-65d77653426b"
version = "0.4.2"

[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[deps.CEnum]]
git-tree-sha1 = "eb4cb44a499229b3b8426dcfb5dd85333951ff90"
uuid = "fa961155-64e5-5f13-b03f-caf6b980ea82"
version = "0.4.2"

[[deps.CUDA]]
deps = ["AbstractFFTs", "Adapt", "BFloat16s", "CEnum", "CUDA_Driver_jll", "CUDA_Runtime_Discovery", "CUDA_Runtime_jll", "CompilerSupportLibraries_jll", "ExprTools", "GPUArrays", "GPUCompiler", "LLVM", "LazyArtifacts", "Libdl", "LinearAlgebra", "Logging", "Preferences", "Printf", "Random", "Random123", "RandomNumbers", "Reexport", "Requires", "SparseArrays", "SpecialFunctions"]
git-tree-sha1 = "cdfda8c9e3fc38116a74807db9d74cbd2ebafd4e"
uuid = "052768ef-5323-5732-b1bb-66c8b64840ba"
version = "4.0.0"

[[deps.CUDA_Driver_jll]]
deps = ["Artifacts", "JLLWrappers", "LazyArtifacts", "Libdl", "Pkg"]
git-tree-sha1 = "75d7896d1ec079ef10d3aee8f3668c11354c03a1"
uuid = "4ee394cb-3365-5eb0-8335-949819d2adfc"
version = "0.2.0+0"

[[deps.CUDA_Runtime_Discovery]]
deps = ["Libdl"]
git-tree-sha1 = "58dd8ec29f54f08c04b052d2c2fa6760b4f4b3a4"
uuid = "1af6417a-86b4-443c-805f-a4643ffb695f"
version = "0.1.1"

[[deps.CUDA_Runtime_jll]]
deps = ["Artifacts", "CUDA_Driver_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "Pkg", "TOML"]
git-tree-sha1 = "d3e6ccd30f84936c1a3a53d622d85d7d3f9b9486"
uuid = "76a88914-d11a-5bdc-97e0-2f5a05c973a2"
version = "0.2.3+2"

[[deps.ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "c6d890a52d2c4d55d326439580c3b8d0875a77d9"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.15.7"

[[deps.Compat]]
deps = ["Dates", "LinearAlgebra", "UUIDs"]
git-tree-sha1 = "61fdd77467a5c3ad071ef8277ac6bd6af7dd4c04"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "4.6.0"

[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"
version = "1.0.2+0"

[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[deps.DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "2fb1e02f2b635d0845df5d7c167fec4dd739b00d"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.9.3"

[[deps.Downloads]]
deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
version = "1.6.0"

[[deps.ExprTools]]
git-tree-sha1 = "56559bbef6ca5ea0c0818fa5c90320398a6fbf8d"
uuid = "e2ba6199-217a-4e67-a87a-7c52f15ade04"
version = "0.1.8"

[[deps.FileWatching]]
uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee"

[[deps.GPUArrays]]
deps = ["Adapt", "GPUArraysCore", "LLVM", "LinearAlgebra", "Printf", "Random", "Reexport", "Serialization", "Statistics"]
git-tree-sha1 = "4dfaff044eb2ce11a897fecd85538310e60b91e6"
uuid = "0c68f7d7-f131-5f86-a1c3-88cf8149b2d7"
version = "8.6.2"

[[deps.GPUArraysCore]]
deps = ["Adapt"]
git-tree-sha1 = "57f7cde02d7a53c9d1d28443b9f11ac5fbe7ebc9"
uuid = "46192b85-c4d5-4398-a991-12ede77f4527"
version = "0.1.3"

[[deps.GPUCompiler]]
deps = ["ExprTools", "InteractiveUtils", "LLVM", "Libdl", "Logging", "TimerOutputs", "UUIDs"]
git-tree-sha1 = "48832a7cacbe56e591a7bef690c78b9d00bcc692"
uuid = "61eb1bfa-7361-4325-ad38-22787b887f55"
version = "0.17.1"

[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[deps.IrrationalConstants]]
git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.1"

[[deps.JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.4.1"

[[deps.LLVM]]
deps = ["CEnum", "LLVMExtra_jll", "Libdl", "Printf", "Unicode"]
git-tree-sha1 = "b8ae281340f0d3e973aae7b96fb7502b0119b376"
uuid = "929cbde3-209d-540e-8aea-75f648917ca0"
version = "4.15.0"

[[deps.LLVMExtra_jll]]
deps = ["Artifacts", "JLLWrappers", "LazyArtifacts", "Libdl", "Pkg", "TOML"]
git-tree-sha1 = "771bfe376249626d3ca12bcd58ba243d3f961576"
uuid = "dad2f222-ce93-54a1-a47d-0025e8a3acab"
version = "0.0.16+0"

[[deps.LazyArtifacts]]
deps = ["Artifacts", "Pkg"]
uuid = "4af54fe1-eca0-43a8-85a7-787d91b784e3"

[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
version = "0.6.3"

[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
version = "7.84.0+0"

[[deps.LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
version = "1.10.2+0"

[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[deps.LinearAlgebra]]
deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[deps.LogExpFunctions]]
deps = ["DocStringExtensions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "45b288af6956e67e621c5cbb2d75a261ab58300b"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.20"

    [deps.LogExpFunctions.extensions]
    ChainRulesCoreExt = "ChainRulesCore"
    ChangesOfVariablesExt = "ChangesOfVariables"
    InverseFunctionsExt = "InverseFunctions"

    [deps.LogExpFunctions.weakdeps]
    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
    ChangesOfVariables = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
    InverseFunctions = "3587e190-3f89-42d0-90ee-14403ec27112"

[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
version = "2.28.0+0"

[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
version = "2022.10.11"

[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
version = "1.2.0"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"
version = "0.3.21+0"

[[deps.OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"
version = "0.8.1+0"

[[deps.OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.5+0"

[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
version = "1.9.0"

[[deps.Preferences]]
deps = ["TOML"]
git-tree-sha1 = "47e5f437cc0e7ef2ce8406ce1e7e24d44915f88d"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.3.0"

[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[deps.Random]]
deps = ["SHA", "Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[deps.Random123]]
deps = ["Random", "RandomNumbers"]
git-tree-sha1 = "7a1a306b72cfa60634f03a911405f4e64d1b718b"
uuid = "74087812-796a-5b5d-8853-05524746bad3"
version = "1.6.0"

[[deps.RandomNumbers]]
deps = ["Random", "Requires"]
git-tree-sha1 = "043da614cc7e95c703498a491e2c21f58a2b8111"
uuid = "e6cf234a-135c-5ec9-84dd-332b85af5143"
version = "1.5.3"

[[deps.Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[deps.Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
version = "0.7.0"

[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[deps.SparseArrays]]
deps = ["Libdl", "LinearAlgebra", "Random", "Serialization", "SuiteSparse_jll"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[deps.SpecialFunctions]]
deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
git-tree-sha1 = "d75bda01f8c31ebb72df80a46c88b25d1c79c56d"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "2.1.7"

[[deps.Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
version = "1.9.0"

[[deps.SuiteSparse_jll]]
deps = ["Artifacts", "Libdl", "Pkg", "libblastrampoline_jll"]
uuid = "bea87d4a-7f5b-5778-9afe-8cc45184846c"
version = "5.10.1+0"

[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
version = "1.0.3"

[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
version = "1.10.0"

[[deps.Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[deps.TimerOutputs]]
deps = ["ExprTools", "Printf"]
git-tree-sha1 = "f2fd3f288dfc6f507b0c3a2eb3bac009251e548b"
uuid = "a759f4b9-e2f1-59dc-863e-4aeb61b1ea8f"
version = "0.5.22"

[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
version = "1.2.13+0"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
version = "5.2.0+0"

[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
version = "1.48.0+0"

[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
version = "17.4.0+0"

Version info

Details on Julia:

Julia Version 1.9.0-beta3
Commit 24204a73447 (2023-01-18 07:20 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 24 × Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, broadwell)
  Threads: 1 on 24 virtual cores
Environment:
  LD_LIBRARY_PATH = /share/apps/julia/julia-1.9/beta3/lib64:/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/software/git/lib64:/usr/local/cuda-11.2/lib64:/home/feist/.local/modules/openmpi/4.1.0-intel_cuda/lib:/usr/local/software/git/lib64:/usr/local/cuda-11.2/lib64:/opt/intel/oneapi/mpi/2021.1.1/libfabric/lib:/opt/intel/oneapi/mpi/2021.1.1/lib/release:/opt/intel/oneapi/mpi/2021.1.1/lib:/opt/intel/oneapi/tbb/latest/lib/intel64/gcc4.8:/opt/intel/oneapi/ipp/2021.1.1/lib/intel64:/opt/intel/oneapi/ccl/2021.1.1/lib/cpu_gpu_dpcpp:/opt/intel/oneapi/debugger/10.0.0/dep/lib:/opt/intel/oneapi/debugger/10.0.0/libipt/intel64/lib:/opt/intel/oneapi/debugger/10.0.0/gdb/intel64/lib:/opt/intel/oneapi/vpl/2021.1.1/lib:/opt/intel/oneapi/itac/2021.1.1/slib:/opt/intel/oneapi/mkl/latest/lib/intel64:/opt/intel/oneapi/dnnl/2021.1.1/cpu_dpcpp_gpu_dpcpp/lib:/opt/intel/oneapi/ippcp/2021.1.1/lib/intel64:/opt/intel/oneapi/dal/2021.1.1/lib/intel64:/opt/intel/oneapi/compiler/2021.1.2/linux/lib:/opt/intel/oneapi/compiler/2021.1.2/linux/lib/x64:/opt/intel/oneapi/compiler/2021.1.2/linux/lib/emu:/opt/intel/oneapi/compiler/2021.1.2/linux/lib/oclfpga/host/linux64/lib:/opt/intel/oneapi/compiler/2021.1.2/linux/lib/oclfpga/linux64/lib:/opt/intel/oneapi/compiler/2021.1.2/linux/compiler/lib/intel64_lin:/opt/intel/oneapi/compiler/2021.1.2/linux/compiler/lib:/usr/local/software/git/lib64:/usr/local/cuda-11.2/lib64:
  LD_GOLD = /share/apps/anaconda/ju2/bin/x86_64-conda-linux-gnu-ld.gold
  JULIA_DEPOT_PATH = :/share/apps/julia/julia-1.9/depot

Details on CUDA:

CUDA runtime 11.8, artifact installation
CUDA driver 11.2
NVIDIA driver 460.39.0

Libraries: 
- CUBLAS: 11.3.1
- CURAND: 10.3.0
- CUFFT: 10.9.0
- CUSOLVER: 11.4.1
- CUSPARSE: 11.7.5
- CUPTI: 18.0.0
- NVML: 11.0.0+460.39

Toolchain:
- Julia: 1.9.0-beta3
- LLVM: 14.0.6
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4, 6.5, 7.0, 7.1, 7.2
- Device capability support: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86

1 device:
  0: GeForce GTX 1080 Ti (sm_61, 10.551 GiB / 10.917 GiB available)

Additional context

I checked that the master branch also shows the error, while everything works fine in CUDA v3.13.0. The error also shows up with any other element type I checked (Float32, ComplexF64, etc).

@jfeist jfeist added the bug Something isn't working label Feb 4, 2023
@jfeist
Copy link
Author

jfeist commented Feb 5, 2023

I just realized that this was due to there being a system-wide CUDA installation in LD_LIBRARY_PATH. Removing that fixes the problem, so it seems that it was using the system-wide CUDA instead of the one from the artifact. So it's not really a bug in CUDA.jl. Still, is there some way to ensure that CUDA.jl picks up the artifact CUDA even if a system-wide CUDA is in the path?

@maleadt
Copy link
Member

maleadt commented Feb 6, 2023

I just realized that this was due to there being a system-wide CUDA installation in LD_LIBRARY_PATH. Removing that fixes the problem, so it seems that it was using the system-wide CUDA instead of the one from the artifact. So it's not really a bug in CUDA.jl.

That is unlikely, and is not how JLLs work. Even if you set a LD_LIBRARY_PATH, libcublas will be resolved to the one from the JLL:

❯ fd libcublas /opt
/opt/cuda/targets/x86_64-linux/lib/libcublas.so

❯ LD_LIBRARY_PATH=/opt/cuda/targets/x86_64-linux/lib/ jl +1.8 --project
julia> using CUDA

julia> CUDA.libcublas
"/home/tim/Julia/depot/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so"

Could you run both scenarios (i.e., the working one without LD_LIBRARY_PATH and the broken one with LD_LIBRARY_PATH set to a system-wide installation) while LD_DEBUG is set to libs, and post the output here?

@maleadt maleadt changed the title norm(::CuArray) fails in v4.0.0 System-wide CUDA in LD_LIBRARY_PATH breaks CUBLAS Feb 6, 2023
@maleadt maleadt added the needs information Further information is requested label Feb 6, 2023
@jfeist
Copy link
Author

jfeist commented Feb 6, 2023

Here it is, hope it helps!

Output with system CUDA in LD_LIBRARY_PATH

> echo $LD_LIBRARY_PATH
/share/apps/julia/julia-1.9/beta3/lib64:/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:
> export LD_DEBUG=libs
> julia --project=. -e 'using CUDA, LinearAlgebra; println(norm(CUDA.fill(1.,1)))'
     22457:	find library=libdl.so.2 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib64/tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib64/tls:/share/apps/julia/julia-1.9/beta3/lib64/x86_64:/share/apps/julia/julia-1.9/beta3/lib64:/share/apps/julia/julia-1.9/beta3/lib/tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/tls:/share/apps/julia/julia-1.9/beta3/lib/x86_64:/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64/tls/x86_64:/usr/local/cuda/lib64/tls:/usr/local/cuda/lib64/x86_64:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64/tls/x86_64:/usr/local/cuda-11.2/lib64/tls:/usr/local/cuda-11.2/lib64/x86_64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/tls/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/tls/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/tls/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/tls/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libdl.so.2
     22457:	  trying file=/usr/local/cuda/lib64/tls/x86_64/libdl.so.2
     22457:	  trying file=/usr/local/cuda/lib64/tls/libdl.so.2
     22457:	  trying file=/usr/local/cuda/lib64/x86_64/libdl.so.2
     22457:	  trying file=/usr/local/cuda/lib64/libdl.so.2
     22457:	  trying file=/usr/local/cuda-11.2/lib64/tls/x86_64/libdl.so.2
     22457:	  trying file=/usr/local/cuda-11.2/lib64/tls/libdl.so.2
     22457:	  trying file=/usr/local/cuda-11.2/lib64/x86_64/libdl.so.2
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libdl.so.2
     22457:	  trying file=tls/x86_64/libdl.so.2
     22457:	  trying file=tls/libdl.so.2
     22457:	  trying file=x86_64/libdl.so.2
     22457:	  trying file=libdl.so.2
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/bin/../lib/tls/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib/tls:/share/apps/julia/julia-1.9/beta3/bin/../lib/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia	(RUNPATH from file julia)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/tls/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/tls/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/x86_64/libdl.so.2
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/libdl.so.2
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libdl.so.2
     22457:
     22457:	find library=libpthread.so.0 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libpthread.so.0
     22457:	  trying file=/usr/local/cuda/lib64/libpthread.so.0
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libpthread.so.0
     22457:	  trying file=tls/x86_64/libpthread.so.0
     22457:	  trying file=tls/libpthread.so.0
     22457:	  trying file=x86_64/libpthread.so.0
     22457:	  trying file=libpthread.so.0
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/bin/../lib:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia		(RUNPATH from file julia)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/libpthread.so.0
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/libpthread.so.0
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libpthread.so.0
     22457:
     22457:	find library=libc.so.6 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libc.so.6
     22457:	  trying file=/usr/local/cuda/lib64/libc.so.6
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libc.so.6
     22457:	  trying file=tls/x86_64/libc.so.6
     22457:	  trying file=tls/libc.so.6
     22457:	  trying file=x86_64/libc.so.6
     22457:	  trying file=libc.so.6
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/bin/../lib:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia		(RUNPATH from file julia)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/libc.so.6
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/libc.so.6
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libc.so.6
     22457:
     22457:	find library=libjulia.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libjulia.so.1
     22457:
     22457:
     22457:	calling init: /lib64/libpthread.so.0
     22457:
     22457:
     22457:	calling init: /lib64/libc.so.6
     22457:
     22457:
     22457:	calling init: /lib64/libdl.so.2
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/libjulia.so.1
     22457:
     22459:	find library=libstdc++.so.6 [0]; searching
     22459:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22459:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libstdc++.so.6
     22459:	  trying file=/usr/local/cuda/lib64/libstdc++.so.6
     22459:	  trying file=/usr/local/cuda-11.2/lib64/libstdc++.so.6
     22459:	  trying file=tls/x86_64/libstdc++.so.6
     22459:	  trying file=tls/libstdc++.so.6
     22459:	  trying file=x86_64/libstdc++.so.6
     22459:	  trying file=libstdc++.so.6
     22459:	 search cache=/etc/ld.so.cache
     22459:	  trying file=/lib64/libstdc++.so.6
     22459:
     22459:	find library=libm.so.6 [0]; searching
     22459:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22459:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libm.so.6
     22459:	  trying file=/usr/local/cuda/lib64/libm.so.6
     22459:	  trying file=/usr/local/cuda-11.2/lib64/libm.so.6
     22459:	  trying file=tls/x86_64/libm.so.6
     22459:	  trying file=tls/libm.so.6
     22459:	  trying file=x86_64/libm.so.6
     22459:	  trying file=libm.so.6
     22459:	 search cache=/etc/ld.so.cache
     22459:	  trying file=/lib64/libm.so.6
     22459:
     22459:	find library=libgcc_s.so.1 [0]; searching
     22459:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22459:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgcc_s.so.1
     22459:	  trying file=/usr/local/cuda/lib64/libgcc_s.so.1
     22459:	  trying file=/usr/local/cuda-11.2/lib64/libgcc_s.so.1
     22459:	  trying file=tls/x86_64/libgcc_s.so.1
     22459:	  trying file=tls/libgcc_s.so.1
     22459:	  trying file=x86_64/libgcc_s.so.1
     22459:	  trying file=libgcc_s.so.1
     22459:	 search cache=/etc/ld.so.cache
     22459:	  trying file=/lib64/libgcc_s.so.1
     22459:
     22459:
     22459:	calling init: /lib64/libgcc_s.so.1
     22459:
     22459:
     22459:	calling init: /lib64/libm.so.6
     22459:
     22459:
     22459:	calling init: /lib64/libstdc++.so.6
     22459:
     22459:	/lib64/libstdc++.so.6: error: symbol lookup error: undefined symbol: GLIBCXX_3.4.31 (fatal)
     22457:	find library=libstdc++.so.6 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libstdc++.so.6
     22457:	  trying file=/usr/local/cuda/lib64/libstdc++.so.6
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libstdc++.so.6
     22457:	  trying file=tls/x86_64/libstdc++.so.6
     22457:	  trying file=tls/libstdc++.so.6
     22457:	  trying file=x86_64/libstdc++.so.6
     22457:	  trying file=libstdc++.so.6
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libstdc++.so.6
     22457:
     22457:	find library=libm.so.6 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libm.so.6
     22457:	  trying file=/usr/local/cuda/lib64/libm.so.6
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libm.so.6
     22457:	  trying file=tls/x86_64/libm.so.6
     22457:	  trying file=tls/libm.so.6
     22457:	  trying file=x86_64/libm.so.6
     22457:	  trying file=libm.so.6
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia/tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/tls:/share/apps/julia/julia-1.9/beta3/lib/julia/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia	(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/tls/x86_64/libm.so.6
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/tls/libm.so.6
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/x86_64/libm.so.6
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libm.so.6
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libm.so.6
     22457:
     22457:	find library=libgcc_s.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgcc_s.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libgcc_s.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libgcc_s.so.1
     22457:	  trying file=tls/x86_64/libgcc_s.so.1
     22457:	  trying file=tls/libgcc_s.so.1
     22457:	  trying file=x86_64/libgcc_s.so.1
     22457:	  trying file=libgcc_s.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgcc_s.so.1
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgcc_s.so.1
     22457:
     22457:
     22457:	calling init: /lib64/libm.so.6
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6
     22457:
     22457:	find library=libopenlibm.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libopenlibm.so
     22457:	  trying file=/usr/local/cuda/lib64/libopenlibm.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libopenlibm.so
     22457:	  trying file=tls/x86_64/libopenlibm.so
     22457:	  trying file=tls/libopenlibm.so
     22457:	  trying file=x86_64/libopenlibm.so
     22457:	  trying file=libopenlibm.so
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/x86_64/libopenlibm.so
     22457:	  trying file=/lib64/tls/libopenlibm.so
     22457:	  trying file=/lib64/x86_64/libopenlibm.so
     22457:	  trying file=/lib64/libopenlibm.so
     22457:	  trying file=/usr/lib64/tls/x86_64/libopenlibm.so
     22457:	  trying file=/usr/lib64/tls/libopenlibm.so
     22457:	  trying file=/usr/lib64/x86_64/libopenlibm.so
     22457:	  trying file=/usr/lib64/libopenlibm.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenlibm.so
     22457:
     22457:	find library=libjulia-internal.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libjulia-internal.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libjulia-internal.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libjulia-internal.so.1
     22457:	  trying file=tls/x86_64/libjulia-internal.so.1
     22457:	  trying file=tls/libjulia-internal.so.1
     22457:	  trying file=x86_64/libjulia-internal.so.1
     22457:	  trying file=libjulia-internal.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libjulia-internal.so.1
     22457:	  trying file=/lib64/libjulia-internal.so.1
     22457:	  trying file=/usr/lib64/tls/libjulia-internal.so.1
     22457:	  trying file=/usr/lib64/libjulia-internal.so.1
     22457:
     22457:	find library=libunwind.so.8 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libunwind.so.8
     22457:	  trying file=/usr/local/cuda/lib64/libunwind.so.8
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libunwind.so.8
     22457:	  trying file=tls/x86_64/libunwind.so.8
     22457:	  trying file=tls/libunwind.so.8
     22457:	  trying file=x86_64/libunwind.so.8
     22457:	  trying file=libunwind.so.8
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls:/share/apps/julia/julia-1.9/beta3/lib/julia/../x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libunwind.so.8
     22457:
     22457:	find library=librt.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/librt.so.1
     22457:	  trying file=/usr/local/cuda/lib64/librt.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/librt.so.1
     22457:	  trying file=tls/x86_64/librt.so.1
     22457:	  trying file=tls/librt.so.1
     22457:	  trying file=x86_64/librt.so.1
     22457:	  trying file=librt.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls:/share/apps/julia/julia-1.9/beta3/lib/julia/../x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/librt.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/x86_64/librt.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/librt.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../x86_64/librt.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../librt.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/librt.so.1
     22457:
     22457:	find library=libz.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libz.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libz.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libz.so.1
     22457:	  trying file=tls/x86_64/libz.so.1
     22457:	  trying file=tls/libz.so.1
     22457:	  trying file=x86_64/libz.so.1
     22457:	  trying file=libz.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libz.so.1
     22457:
     22457:	find library=libatomic.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libatomic.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libatomic.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libatomic.so.1
     22457:	  trying file=tls/x86_64/libatomic.so.1
     22457:	  trying file=tls/libatomic.so.1
     22457:	  trying file=x86_64/libatomic.so.1
     22457:	  trying file=libatomic.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libatomic.so.1
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libatomic.so.1
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libz.so.1
     22457:
     22457:
     22457:	calling init: /lib64/librt.so.1
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1
     22457:
     22457:	find library=libjulia-codegen.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libjulia-codegen.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libjulia-codegen.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libjulia-codegen.so.1
     22457:	  trying file=tls/x86_64/libjulia-codegen.so.1
     22457:	  trying file=tls/libjulia-codegen.so.1
     22457:	  trying file=x86_64/libjulia-codegen.so.1
     22457:	  trying file=libjulia-codegen.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libjulia-codegen.so.1
     22457:	  trying file=/lib64/libjulia-codegen.so.1
     22457:	  trying file=/usr/lib64/tls/libjulia-codegen.so.1
     22457:	  trying file=/usr/lib64/libjulia-codegen.so.1
     22457:
     22457:	find library=libLLVM-14jl.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libLLVM-14jl.so
     22457:	  trying file=/usr/local/cuda/lib64/libLLVM-14jl.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libLLVM-14jl.so
     22457:	  trying file=tls/x86_64/libLLVM-14jl.so
     22457:	  trying file=tls/libLLVM-14jl.so
     22457:	  trying file=x86_64/libLLVM-14jl.so
     22457:	  trying file=libLLVM-14jl.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libLLVM-14jl.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libLLVM-14jl.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-codegen.so.1
     22457:
     22457:
     22457:	initialize program: julia
     22457:
     22457:
     22457:	transferring control: julia
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/sys.so
     22457:
     22457:	find library=libpcre2-8 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libpcre2-8
     22457:	  trying file=/usr/local/cuda/lib64/libpcre2-8
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libpcre2-8
     22457:	  trying file=tls/x86_64/libpcre2-8
     22457:	  trying file=tls/libpcre2-8
     22457:	  trying file=x86_64/libpcre2-8
     22457:	  trying file=libpcre2-8
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libpcre2-8
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libpcre2-8
     22457:	  trying file=/lib64/libpcre2-8
     22457:	  trying file=/usr/lib64/tls/libpcre2-8
     22457:	  trying file=/usr/lib64/libpcre2-8
     22457:
     22457:	find library=libpcre2-8.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libpcre2-8.so
     22457:	  trying file=/usr/local/cuda/lib64/libpcre2-8.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libpcre2-8.so
     22457:	  trying file=tls/x86_64/libpcre2-8.so
     22457:	  trying file=tls/libpcre2-8.so
     22457:	  trying file=x86_64/libpcre2-8.so
     22457:	  trying file=libpcre2-8.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8.so
     22457:
     22457:	find library=libgmp [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgmp
     22457:	  trying file=/usr/local/cuda/lib64/libgmp
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libgmp
     22457:	  trying file=tls/x86_64/libgmp
     22457:	  trying file=tls/libgmp
     22457:	  trying file=x86_64/libgmp
     22457:	  trying file=libgmp
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgmp
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libgmp
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libgmp
     22457:	  trying file=/lib64/libgmp
     22457:	  trying file=/usr/lib64/tls/libgmp
     22457:	  trying file=/usr/lib64/libgmp
     22457:
     22457:	find library=libgmp.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgmp.so
     22457:	  trying file=/usr/local/cuda/lib64/libgmp.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libgmp.so
     22457:	  trying file=tls/x86_64/libgmp.so
     22457:	  trying file=tls/libgmp.so
     22457:	  trying file=x86_64/libgmp.so
     22457:	  trying file=libgmp.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgmp.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgmp.so
     22457:
     22457:	find library=libmpfr [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmpfr
     22457:	  trying file=/usr/local/cuda/lib64/libmpfr
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libmpfr
     22457:	  trying file=tls/x86_64/libmpfr
     22457:	  trying file=tls/libmpfr
     22457:	  trying file=x86_64/libmpfr
     22457:	  trying file=libmpfr
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libmpfr
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libmpfr
     22457:	  trying file=/lib64/libmpfr
     22457:	  trying file=/usr/lib64/tls/libmpfr
     22457:	  trying file=/usr/lib64/libmpfr
     22457:
     22457:	find library=libmpfr.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmpfr.so
     22457:	  trying file=/usr/local/cuda/lib64/libmpfr.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libmpfr.so
     22457:	  trying file=tls/x86_64/libmpfr.so
     22457:	  trying file=tls/libmpfr.so
     22457:	  trying file=x86_64/libmpfr.so
     22457:	  trying file=libmpfr.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr.so
     22457:
     22457:	find library=libgfortran.so.5 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgfortran.so.5
     22457:	  trying file=/usr/local/cuda/lib64/libgfortran.so.5
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libgfortran.so.5
     22457:	  trying file=tls/x86_64/libgfortran.so.5
     22457:	  trying file=tls/libgfortran.so.5
     22457:	  trying file=x86_64/libgfortran.so.5
     22457:	  trying file=libgfortran.so.5
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgfortran.so.5
     22457:
     22457:	find library=libquadmath.so.0 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libquadmath.so.0
     22457:	  trying file=/usr/local/cuda/lib64/libquadmath.so.0
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libquadmath.so.0
     22457:	  trying file=tls/x86_64/libquadmath.so.0
     22457:	  trying file=tls/libquadmath.so.0
     22457:	  trying file=x86_64/libquadmath.so.0
     22457:	  trying file=libquadmath.so.0
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libquadmath.so.0
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libquadmath.so.0
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgfortran.so.5
     22457:
     22457:	find library=libgomp.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgomp.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libgomp.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libgomp.so.1
     22457:	  trying file=tls/x86_64/libgomp.so.1
     22457:	  trying file=tls/libgomp.so.1
     22457:	  trying file=x86_64/libgomp.so.1
     22457:	  trying file=libgomp.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgomp.so.1
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgomp.so.1
     22457:
     22457:	find library=libssp.so.0 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libssp.so.0
     22457:	  trying file=/usr/local/cuda/lib64/libssp.so.0
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libssp.so.0
     22457:	  trying file=tls/x86_64/libssp.so.0
     22457:	  trying file=tls/libssp.so.0
     22457:	  trying file=x86_64/libssp.so.0
     22457:	  trying file=libssp.so.0
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libssp.so.0
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libssp.so.0
     22457:
     22457:	find library=libopenblas64_.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libopenblas64_.so
     22457:	  trying file=/usr/local/cuda/lib64/libopenblas64_.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libopenblas64_.so
     22457:	  trying file=tls/x86_64/libopenblas64_.so
     22457:	  trying file=tls/libopenblas64_.so
     22457:	  trying file=x86_64/libopenblas64_.so
     22457:	  trying file=libopenblas64_.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so
     22457:
     22457:	find library=libblastrampoline.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libblastrampoline.so
     22457:	  trying file=/usr/local/cuda/lib64/libblastrampoline.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libblastrampoline.so
     22457:	  trying file=tls/x86_64/libblastrampoline.so
     22457:	  trying file=tls/libblastrampoline.so
     22457:	  trying file=x86_64/libblastrampoline.so
     22457:	  trying file=libblastrampoline.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libblastrampoline.so
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libblastrampoline.so
     22457:
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax__ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax___ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax_64 (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_dbf16tod64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbdot64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbdtobf1664_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbf16tos64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbgemm64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbgemv64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbstobf1664_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: dbf16tod_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qamax_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qamin_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qasum_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qaxpy_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qcabs1_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qcopy_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qdot_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qgbmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qgemm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qgemv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qger_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qmax_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qmin_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qnrm2_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsbmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qscal_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qspmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qspr2_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qspr_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qswap_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsymm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsymv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyr2_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyr2k_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyr_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyrk_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtbmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtbsv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtpmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtpsv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrmm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrsm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrsv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxamax_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxamin_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxasum_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxnrm2_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbdot_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbdtobf16_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbf16tos_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbgemm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbgemv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbstobf16_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xaxpy_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xcopy_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xdotc_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xdotu_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgbmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgemm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgemv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgerc_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgeru_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhbmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhemm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhemv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xher2_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xher2k_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xher_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xherk_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhpmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhpr2_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhpr_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xqscal_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xscal_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xswap_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xsymm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xsyr2k_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xsyrk_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtbmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtbsv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtpmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtpsv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrmm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrmv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrsm_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrsv_64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: MKL_Set_Num_Threads64_ (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: bli_thread_set_num_threads64_ (fatal)
     22457:	find library=libcholmod [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcholmod
     22457:	  trying file=/usr/local/cuda/lib64/libcholmod
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcholmod
     22457:	  trying file=tls/x86_64/libcholmod
     22457:	  trying file=tls/libcholmod
     22457:	  trying file=x86_64/libcholmod
     22457:	  trying file=libcholmod
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcholmod
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libcholmod
     22457:	  trying file=/lib64/libcholmod
     22457:	  trying file=/usr/lib64/tls/libcholmod
     22457:	  trying file=/usr/lib64/libcholmod
     22457:
     22457:	find library=libcholmod.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcholmod.so
     22457:	  trying file=/usr/local/cuda/lib64/libcholmod.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcholmod.so
     22457:	  trying file=tls/x86_64/libcholmod.so
     22457:	  trying file=tls/libcholmod.so
     22457:	  trying file=x86_64/libcholmod.so
     22457:	  trying file=libcholmod.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod.so
     22457:
     22457:	find library=libamd.so.2 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libamd.so.2
     22457:
     22457:	find library=libcolamd.so.2 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcolamd.so.2
     22457:
     22457:	find library=libsuitesparseconfig.so.5 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so.5
     22457:
     22457:	find library=libccolamd.so.2 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libccolamd.so.2
     22457:
     22457:	find library=libcamd.so.2 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcamd.so.2
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so.5
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcamd.so.2
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libccolamd.so.2
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcolamd.so.2
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libamd.so.2
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod.so
     22457:
     22457:	find library=libcholmod [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcholmod
     22457:	  trying file=/usr/local/cuda/lib64/libcholmod
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcholmod
     22457:	  trying file=tls/x86_64/libcholmod
     22457:	  trying file=tls/libcholmod
     22457:	  trying file=x86_64/libcholmod
     22457:	  trying file=libcholmod
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcholmod
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libcholmod
     22457:	  trying file=/lib64/libcholmod
     22457:	  trying file=/usr/lib64/tls/libcholmod
     22457:	  trying file=/usr/lib64/libcholmod
     22457:
     22457:	find library=libsuitesparseconfig [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libsuitesparseconfig
     22457:	  trying file=/usr/local/cuda/lib64/libsuitesparseconfig
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libsuitesparseconfig
     22457:	  trying file=tls/x86_64/libsuitesparseconfig
     22457:	  trying file=tls/libsuitesparseconfig
     22457:	  trying file=x86_64/libsuitesparseconfig
     22457:	  trying file=libsuitesparseconfig
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libsuitesparseconfig
     22457:	 search cache=/etc/ld.so.cache
     22457:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22457:	  trying file=/lib64/tls/libsuitesparseconfig
     22457:	  trying file=/lib64/libsuitesparseconfig
     22457:	  trying file=/usr/lib64/tls/libsuitesparseconfig
     22457:	  trying file=/usr/lib64/libsuitesparseconfig
     22457:
     22457:	find library=libsuitesparseconfig.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libsuitesparseconfig.so
     22457:	  trying file=/usr/local/cuda/lib64/libsuitesparseconfig.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libsuitesparseconfig.so
     22457:	  trying file=tls/x86_64/libsuitesparseconfig.so
     22457:	  trying file=tls/libsuitesparseconfig.so
     22457:	  trying file=x86_64/libsuitesparseconfig.so
     22457:	  trying file=libsuitesparseconfig.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so
     22457:
     22457:	find library=libnghttp2.so.14 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libnghttp2.so.14
     22457:	  trying file=/usr/local/cuda/lib64/libnghttp2.so.14
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libnghttp2.so.14
     22457:	  trying file=tls/x86_64/libnghttp2.so.14
     22457:	  trying file=tls/libnghttp2.so.14
     22457:	  trying file=x86_64/libnghttp2.so.14
     22457:	  trying file=libnghttp2.so.14
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libnghttp2.so.14
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libnghttp2.so.14
     22457:
     22457:	find library=libcurl.so [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcurl.so
     22457:	  trying file=/usr/local/cuda/lib64/libcurl.so
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcurl.so
     22457:	  trying file=tls/x86_64/libcurl.so
     22457:	  trying file=tls/libcurl.so
     22457:	  trying file=x86_64/libcurl.so
     22457:	  trying file=libcurl.so
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcurl.so
     22457:
     22457:	find library=libssh2.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libssh2.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libssh2.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libssh2.so.1
     22457:	  trying file=tls/x86_64/libssh2.so.1
     22457:	  trying file=tls/libssh2.so.1
     22457:	  trying file=x86_64/libssh2.so.1
     22457:	  trying file=libssh2.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libssh2.so.1
     22457:
     22457:	find library=libmbedtls.so.14 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmbedtls.so.14
     22457:	  trying file=/usr/local/cuda/lib64/libmbedtls.so.14
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libmbedtls.so.14
     22457:	  trying file=tls/x86_64/libmbedtls.so.14
     22457:	  trying file=tls/libmbedtls.so.14
     22457:	  trying file=x86_64/libmbedtls.so.14
     22457:	  trying file=libmbedtls.so.14
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmbedtls.so.14
     22457:
     22457:	find library=libmbedx509.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmbedx509.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libmbedx509.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libmbedx509.so.1
     22457:	  trying file=tls/x86_64/libmbedx509.so.1
     22457:	  trying file=tls/libmbedx509.so.1
     22457:	  trying file=x86_64/libmbedx509.so.1
     22457:	  trying file=libmbedx509.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmbedx509.so.1
     22457:
     22457:	find library=libmbedcrypto.so.7 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmbedcrypto.so.7
     22457:	  trying file=/usr/local/cuda/lib64/libmbedcrypto.so.7
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libmbedcrypto.so.7
     22457:	  trying file=tls/x86_64/libmbedcrypto.so.7
     22457:	  trying file=tls/libmbedcrypto.so.7
     22457:	  trying file=x86_64/libmbedcrypto.so.7
     22457:	  trying file=libmbedcrypto.so.7
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmbedcrypto.so.7
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedcrypto.so.7
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedx509.so.1
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedtls.so.14
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libssh2.so.1
     22457:
     22457:
     22457:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcurl.so
     22457:
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22457:	find library=libcuda.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libcuda.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcuda.so.1
     22457:	  trying file=tls/x86_64/libcuda.so.1
     22457:	  trying file=tls/libcuda.so.1
     22457:	  trying file=x86_64/libcuda.so.1
     22457:	  trying file=libcuda.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libcuda.so.1
     22457:
     22457:
     22457:	calling init: /lib64/libcuda.so.1
     22457:
     22457:
     22457:	calling fini: /lib64/libcuda.so.1 [0]
     22457:
     22457:	find library=libcuda.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libcuda.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcuda.so.1
     22457:	  trying file=tls/x86_64/libcuda.so.1
     22457:	  trying file=tls/libcuda.so.1
     22457:	  trying file=x86_64/libcuda.so.1
     22457:	  trying file=libcuda.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libcuda.so.1
     22457:
     22457:	find library=libcuda.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libcuda.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcuda.so.1
     22457:	  trying file=tls/x86_64/libcuda.so.1
     22457:	  trying file=tls/libcuda.so.1
     22457:	  trying file=x86_64/libcuda.so.1
     22457:	  trying file=libcuda.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libcuda.so.1
     22457:
     22457:
     22457:	calling init: /lib64/libcuda.so.1
     22457:
     22457:
     22457:	calling fini: /lib64/libcuda.so.1 [0]
     22457:
     22457:	find library=libcuda.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libcuda.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcuda.so.1
     22457:	  trying file=tls/x86_64/libcuda.so.1
     22457:	  trying file=tls/libcuda.so.1
     22457:	  trying file=x86_64/libcuda.so.1
     22457:	  trying file=libcuda.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libcuda.so.1
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/b5e755e06f4d49a5ab1a638eea5d75bf20c66e3d/lib/libcuda.so
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/b5e755e06f4d49a5ab1a638eea5d75bf20c66e3d/lib/libcuda.so [0]
     22457:
     22457:	find library=libcublasLt.so.11 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcublasLt.so.11
     22457:	  trying file=/usr/local/cuda/lib64/libcublasLt.so.11
     22457:
     22457:
     22457:	calling init: /usr/local/cuda/lib64/libcublasLt.so.11
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcudart.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcufft.so
     22457:
     22457:	find library=libutil.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libutil.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libutil.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libutil.so.1
     22457:	  trying file=tls/x86_64/libutil.so.1
     22457:	  trying file=tls/libutil.so.1
     22457:	  trying file=x86_64/libutil.so.1
     22457:	  trying file=libutil.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libutil.so.1
     22457:
     22457:
     22457:	calling init: /lib64/libutil.so.1
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcupti.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcurand.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolver.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolverMg.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusparse.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvToolsExt.so
     22457:
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvvm.so
     22457:
     22457:	find library=libcuda.so.1 [0]; searching
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22457:	  trying file=/usr/local/cuda/lib64/libcuda.so.1
     22457:	  trying file=/usr/local/cuda-11.2/lib64/libcuda.so.1
     22457:	  trying file=tls/x86_64/libcuda.so.1
     22457:	  trying file=tls/libcuda.so.1
     22457:	  trying file=x86_64/libcuda.so.1
     22457:	  trying file=libcuda.so.1
     22457:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22457:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22457:	 search cache=/etc/ld.so.cache
     22457:	  trying file=/lib64/libcuda.so.1
     22457:
     22457:
     22457:	calling init: /lib64/libcuda.so.1
     22457:
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuFlushGPUDirectRDMAWrites (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuModuleGetLoadingMode (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuArrayGetMemoryRequirements (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuMipmappedArrayGetMemoryRequirements (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx_ptsz (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxPotentialClusterSize (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxActiveClusters (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemAllocNode (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemAllocNodeGetParams (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemFreeNode (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemFreeNodeGetParams (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGraphMemTrim (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGetGraphMemAttribute (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceSetGraphMemAttribute (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphInstantiateWithFlags (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2 (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2_ptsz (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies_ptsz (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphDebugDotPrint (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectCreate (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRetain (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRelease (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphRetainUserObject (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphReleaseUserObject (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeSetEnabled (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeGetEnabled (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:
     22457:	calling init: /home/feist/.julia/artifacts/9723dd10f25f310e5d2c3f1113ef158db3aa89b2/lib/libLLVMExtra-14.so
     22457:
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_symbol_name (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_symbol_name (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemmove (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemmove (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuFlushGPUDirectRDMAWrites (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuModuleGetLoadingMode (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuArrayGetMemoryRequirements (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuMipmappedArrayGetMemoryRequirements (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx_ptsz (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxPotentialClusterSize (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxActiveClusters (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemAllocNode (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemAllocNodeGetParams (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemFreeNode (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemFreeNodeGetParams (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGraphMemTrim (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGetGraphMemAttribute (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceSetGraphMemAttribute (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphInstantiateWithFlags (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2 (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2_ptsz (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies_ptsz (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphDebugDotPrint (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectCreate (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRetain (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRelease (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphRetainUserObject (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphReleaseUserObject (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeSetEnabled (fatal)
     22457:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeGetEnabled (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
ERROR: DivideError: integer division error
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/libcublas.jl:106 [inlined]
 [2] macro expansion
   @ ~/.julia/packages/CUDA/ZX8mg/src/pool.jl:312 [inlined]
 [3] macro expansion
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/libcublas.jl:22 [inlined]
 [4] cublasDnrm2_v2(handle::Ptr{CUDA.CUBLAS.cublasContext}, n::Int64, x::CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}, incx::Int64, result::Base.RefValue{Float64})
   @ CUDA.CUBLAS ~/.julia/packages/CUDA/ZX8mg/lib/utils/call.jl:26
 [5] nrm2
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/wrappers.jl:168 [inlined]
 [6] nrm2
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/wrappers.jl:173 [inlined]
 [7] norm
   @ ~/.julia/packages/CUDA/ZX8mg/lib/cublas/linalg.jl:108 [inlined]
 [8] norm(x::CuArray{Float64, 1, CUDA.Mem.DeviceBuffer})
   @ CUDA.CUBLAS ~/.julia/packages/CUDA/ZX8mg/lib/cublas/linalg.jl:107
 [9] top-level scope
   @ none:1
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_stdout (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_stdout (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_putb (fatal)
     22457:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_putb (fatal)
     22457:
     22457:	calling fini: julia [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenlibm.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-codegen.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/sys.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgmp.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgomp.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libssp.so.0 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgfortran.so.5 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libquadmath.so.0 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libblastrampoline.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libamd.so.2 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcolamd.so.2 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libccolamd.so.2 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcamd.so.2 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so.5 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcurl.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libnghttp2.so.14 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libssh2.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedtls.so.14 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedx509.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedcrypto.so.7 [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcudart.so [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcufft.so [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcupti.so [0]
     22457:
     22457:
     22457:	calling fini: /lib64/libutil.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcurand.so [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolver.so [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolverMg.so [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so [0]
     22457:
     22457:
     22457:	calling fini: /usr/local/cuda/lib64/libcublasLt.so.11 [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusparse.so [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvToolsExt.so [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvvm.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libatomic.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/libjulia.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /lib64/libcuda.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /home/feist/.julia/artifacts/9723dd10f25f310e5d2c3f1113ef158db3aa89b2/lib/libLLVMExtra-14.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libLLVM-14jl.so [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libz.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /lib64/librt.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6 [0]
     22457:
     22457:
     22457:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgcc_s.so.1 [0]
     22457:
     22457:
     22457:	calling fini: /lib64/libm.so.6 [0]
     22457:
     22457:
     22457:	calling fini: /lib64/libpthread.so.0 [0]
     22457:
     22457:
     22457:	calling fini: /lib64/libdl.so.2 [0]
     22457:

Output without system CUDA in LD_LIBRARY_PATH

> echo $LD_LIBRARY_PATH
/share/apps/julia/julia-1.9/beta3/lib64:/share/apps/julia/julia-1.9/beta3/lib:
> export LD_DEBUG=libs
> julia --project=. -e 'using CUDA, LinearAlgebra; println(norm(CUDA.fill(1.,1)))'
     22480:	find library=libdl.so.2 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib64/tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib64/tls:/share/apps/julia/julia-1.9/beta3/lib64/x86_64:/share/apps/julia/julia-1.9/beta3/lib64:/share/apps/julia/julia-1.9/beta3/lib/tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/tls:/share/apps/julia/julia-1.9/beta3/lib/x86_64:/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/tls/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/tls/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/tls/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/tls/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libdl.so.2
     22480:	  trying file=tls/x86_64/libdl.so.2
     22480:	  trying file=tls/libdl.so.2
     22480:	  trying file=x86_64/libdl.so.2
     22480:	  trying file=libdl.so.2
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/bin/../lib/tls/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib/tls:/share/apps/julia/julia-1.9/beta3/bin/../lib/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/x86_64:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia	(RUNPATH from file julia)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/tls/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/tls/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/tls/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/x86_64/libdl.so.2
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/libdl.so.2
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libdl.so.2
     22480:
     22480:	find library=libpthread.so.0 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libpthread.so.0
     22480:	  trying file=tls/x86_64/libpthread.so.0
     22480:	  trying file=tls/libpthread.so.0
     22480:	  trying file=x86_64/libpthread.so.0
     22480:	  trying file=libpthread.so.0
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/bin/../lib:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia		(RUNPATH from file julia)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/libpthread.so.0
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/libpthread.so.0
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libpthread.so.0
     22480:
     22480:	find library=libc.so.6 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libc.so.6
     22480:	  trying file=tls/x86_64/libc.so.6
     22480:	  trying file=tls/libc.so.6
     22480:	  trying file=x86_64/libc.so.6
     22480:	  trying file=libc.so.6
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/bin/../lib:/share/apps/julia/julia-1.9/beta3/bin/../lib/julia		(RUNPATH from file julia)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/libc.so.6
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/bin/../lib/julia/libc.so.6
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libc.so.6
     22480:
     22480:	find library=libjulia.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libjulia.so.1
     22480:
     22480:
     22480:	calling init: /lib64/libpthread.so.0
     22480:
     22480:
     22480:	calling init: /lib64/libc.so.6
     22480:
     22480:
     22480:	calling init: /lib64/libdl.so.2
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/libjulia.so.1
     22480:
     22481:	find library=libstdc++.so.6 [0]; searching
     22481:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22481:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libstdc++.so.6
     22481:	  trying file=tls/x86_64/libstdc++.so.6
     22481:	  trying file=tls/libstdc++.so.6
     22481:	  trying file=x86_64/libstdc++.so.6
     22481:	  trying file=libstdc++.so.6
     22481:	 search cache=/etc/ld.so.cache
     22481:	  trying file=/lib64/libstdc++.so.6
     22481:
     22481:	find library=libm.so.6 [0]; searching
     22481:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22481:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libm.so.6
     22481:	  trying file=tls/x86_64/libm.so.6
     22481:	  trying file=tls/libm.so.6
     22481:	  trying file=x86_64/libm.so.6
     22481:	  trying file=libm.so.6
     22481:	 search cache=/etc/ld.so.cache
     22481:	  trying file=/lib64/libm.so.6
     22481:
     22481:	find library=libgcc_s.so.1 [0]; searching
     22481:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22481:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgcc_s.so.1
     22481:	  trying file=tls/x86_64/libgcc_s.so.1
     22481:	  trying file=tls/libgcc_s.so.1
     22481:	  trying file=x86_64/libgcc_s.so.1
     22481:	  trying file=libgcc_s.so.1
     22481:	 search cache=/etc/ld.so.cache
     22481:	  trying file=/lib64/libgcc_s.so.1
     22481:
     22481:
     22481:	calling init: /lib64/libgcc_s.so.1
     22481:
     22481:
     22481:	calling init: /lib64/libm.so.6
     22481:
     22481:
     22481:	calling init: /lib64/libstdc++.so.6
     22481:
     22481:	/lib64/libstdc++.so.6: error: symbol lookup error: undefined symbol: GLIBCXX_3.4.31 (fatal)
     22480:	find library=libstdc++.so.6 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libstdc++.so.6
     22480:	  trying file=tls/x86_64/libstdc++.so.6
     22480:	  trying file=tls/libstdc++.so.6
     22480:	  trying file=x86_64/libstdc++.so.6
     22480:	  trying file=libstdc++.so.6
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libstdc++.so.6
     22480:
     22480:	find library=libm.so.6 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libm.so.6
     22480:	  trying file=tls/x86_64/libm.so.6
     22480:	  trying file=tls/libm.so.6
     22480:	  trying file=x86_64/libm.so.6
     22480:	  trying file=libm.so.6
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia/tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/tls:/share/apps/julia/julia-1.9/beta3/lib/julia/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia	(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/tls/x86_64/libm.so.6
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/tls/libm.so.6
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/x86_64/libm.so.6
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libm.so.6
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libm.so.6
     22480:
     22480:	find library=libgcc_s.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgcc_s.so.1
     22480:	  trying file=tls/x86_64/libgcc_s.so.1
     22480:	  trying file=tls/libgcc_s.so.1
     22480:	  trying file=x86_64/libgcc_s.so.1
     22480:	  trying file=libgcc_s.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgcc_s.so.1
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgcc_s.so.1
     22480:
     22480:
     22480:	calling init: /lib64/libm.so.6
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6
     22480:
     22480:	find library=libopenlibm.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libopenlibm.so
     22480:	  trying file=tls/x86_64/libopenlibm.so
     22480:	  trying file=tls/libopenlibm.so
     22480:	  trying file=x86_64/libopenlibm.so
     22480:	  trying file=libopenlibm.so
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/x86_64/libopenlibm.so
     22480:	  trying file=/lib64/tls/libopenlibm.so
     22480:	  trying file=/lib64/x86_64/libopenlibm.so
     22480:	  trying file=/lib64/libopenlibm.so
     22480:	  trying file=/usr/lib64/tls/x86_64/libopenlibm.so
     22480:	  trying file=/usr/lib64/tls/libopenlibm.so
     22480:	  trying file=/usr/lib64/x86_64/libopenlibm.so
     22480:	  trying file=/usr/lib64/libopenlibm.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenlibm.so
     22480:
     22480:	find library=libjulia-internal.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libjulia-internal.so.1
     22480:	  trying file=tls/x86_64/libjulia-internal.so.1
     22480:	  trying file=tls/libjulia-internal.so.1
     22480:	  trying file=x86_64/libjulia-internal.so.1
     22480:	  trying file=libjulia-internal.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libjulia-internal.so.1
     22480:	  trying file=/lib64/libjulia-internal.so.1
     22480:	  trying file=/usr/lib64/tls/libjulia-internal.so.1
     22480:	  trying file=/usr/lib64/libjulia-internal.so.1
     22480:
     22480:	find library=libunwind.so.8 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libunwind.so.8
     22480:	  trying file=tls/x86_64/libunwind.so.8
     22480:	  trying file=tls/libunwind.so.8
     22480:	  trying file=x86_64/libunwind.so.8
     22480:	  trying file=libunwind.so.8
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls:/share/apps/julia/julia-1.9/beta3/lib/julia/../x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libunwind.so.8
     22480:
     22480:	find library=librt.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/librt.so.1
     22480:	  trying file=tls/x86_64/librt.so.1
     22480:	  trying file=tls/librt.so.1
     22480:	  trying file=x86_64/librt.so.1
     22480:	  trying file=librt.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/../tls:/share/apps/julia/julia-1.9/beta3/lib/julia/../x86_64:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/librt.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/x86_64/librt.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../tls/librt.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../x86_64/librt.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../librt.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/librt.so.1
     22480:
     22480:	find library=libz.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libz.so.1
     22480:	  trying file=tls/x86_64/libz.so.1
     22480:	  trying file=tls/libz.so.1
     22480:	  trying file=x86_64/libz.so.1
     22480:	  trying file=libz.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libz.so.1
     22480:
     22480:	find library=libatomic.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libatomic.so.1
     22480:	  trying file=tls/x86_64/libatomic.so.1
     22480:	  trying file=tls/libatomic.so.1
     22480:	  trying file=x86_64/libatomic.so.1
     22480:	  trying file=libatomic.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libatomic.so.1
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libatomic.so.1
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libz.so.1
     22480:
     22480:
     22480:	calling init: /lib64/librt.so.1
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1
     22480:
     22480:	find library=libjulia-codegen.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libjulia-codegen.so.1
     22480:	  trying file=tls/x86_64/libjulia-codegen.so.1
     22480:	  trying file=tls/libjulia-codegen.so.1
     22480:	  trying file=x86_64/libjulia-codegen.so.1
     22480:	  trying file=libjulia-codegen.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libjulia-codegen.so.1
     22480:	  trying file=/lib64/libjulia-codegen.so.1
     22480:	  trying file=/usr/lib64/tls/libjulia-codegen.so.1
     22480:	  trying file=/usr/lib64/libjulia-codegen.so.1
     22480:
     22480:	find library=libLLVM-14jl.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libLLVM-14jl.so
     22480:	  trying file=tls/x86_64/libLLVM-14jl.so
     22480:	  trying file=tls/libLLVM-14jl.so
     22480:	  trying file=x86_64/libLLVM-14jl.so
     22480:	  trying file=libLLVM-14jl.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libLLVM-14jl.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libLLVM-14jl.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-codegen.so.1
     22480:
     22480:
     22480:	initialize program: julia
     22480:
     22480:
     22480:	transferring control: julia
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/sys.so
     22480:
     22480:	find library=libpcre2-8 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libpcre2-8
     22480:	  trying file=tls/x86_64/libpcre2-8
     22480:	  trying file=tls/libpcre2-8
     22480:	  trying file=x86_64/libpcre2-8
     22480:	  trying file=libpcre2-8
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libpcre2-8
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libpcre2-8
     22480:	  trying file=/lib64/libpcre2-8
     22480:	  trying file=/usr/lib64/tls/libpcre2-8
     22480:	  trying file=/usr/lib64/libpcre2-8
     22480:
     22480:	find library=libpcre2-8.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libpcre2-8.so
     22480:	  trying file=tls/x86_64/libpcre2-8.so
     22480:	  trying file=tls/libpcre2-8.so
     22480:	  trying file=x86_64/libpcre2-8.so
     22480:	  trying file=libpcre2-8.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8.so
     22480:
     22480:	find library=libgmp [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgmp
     22480:	  trying file=tls/x86_64/libgmp
     22480:	  trying file=tls/libgmp
     22480:	  trying file=x86_64/libgmp
     22480:	  trying file=libgmp
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgmp
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libgmp
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libgmp
     22480:	  trying file=/lib64/libgmp
     22480:	  trying file=/usr/lib64/tls/libgmp
     22480:	  trying file=/usr/lib64/libgmp
     22480:
     22480:	find library=libgmp.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgmp.so
     22480:	  trying file=tls/x86_64/libgmp.so
     22480:	  trying file=tls/libgmp.so
     22480:	  trying file=x86_64/libgmp.so
     22480:	  trying file=libgmp.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgmp.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgmp.so
     22480:
     22480:	find library=libmpfr [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmpfr
     22480:	  trying file=tls/x86_64/libmpfr
     22480:	  trying file=tls/libmpfr
     22480:	  trying file=x86_64/libmpfr
     22480:	  trying file=libmpfr
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libmpfr
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libmpfr
     22480:	  trying file=/lib64/libmpfr
     22480:	  trying file=/usr/lib64/tls/libmpfr
     22480:	  trying file=/usr/lib64/libmpfr
     22480:
     22480:	find library=libmpfr.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmpfr.so
     22480:	  trying file=tls/x86_64/libmpfr.so
     22480:	  trying file=tls/libmpfr.so
     22480:	  trying file=x86_64/libmpfr.so
     22480:	  trying file=libmpfr.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr.so
     22480:
     22480:	find library=libgfortran.so.5 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgfortran.so.5
     22480:	  trying file=tls/x86_64/libgfortran.so.5
     22480:	  trying file=tls/libgfortran.so.5
     22480:	  trying file=x86_64/libgfortran.so.5
     22480:	  trying file=libgfortran.so.5
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgfortran.so.5
     22480:
     22480:	find library=libquadmath.so.0 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libquadmath.so.0
     22480:	  trying file=tls/x86_64/libquadmath.so.0
     22480:	  trying file=tls/libquadmath.so.0
     22480:	  trying file=x86_64/libquadmath.so.0
     22480:	  trying file=libquadmath.so.0
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libquadmath.so.0
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libquadmath.so.0
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgfortran.so.5
     22480:
     22480:	find library=libgomp.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libgomp.so.1
     22480:	  trying file=tls/x86_64/libgomp.so.1
     22480:	  trying file=tls/libgomp.so.1
     22480:	  trying file=x86_64/libgomp.so.1
     22480:	  trying file=libgomp.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libgomp.so.1
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libgomp.so.1
     22480:
     22480:	find library=libssp.so.0 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libssp.so.0
     22480:	  trying file=tls/x86_64/libssp.so.0
     22480:	  trying file=tls/libssp.so.0
     22480:	  trying file=x86_64/libssp.so.0
     22480:	  trying file=libssp.so.0
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libssp.so.0
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libssp.so.0
     22480:
     22480:	find library=libopenblas64_.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libopenblas64_.so
     22480:	  trying file=tls/x86_64/libopenblas64_.so
     22480:	  trying file=tls/libopenblas64_.so
     22480:	  trying file=x86_64/libopenblas64_.so
     22480:	  trying file=libopenblas64_.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so
     22480:
     22480:	find library=libblastrampoline.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libblastrampoline.so
     22480:	  trying file=tls/x86_64/libblastrampoline.so
     22480:	  trying file=tls/libblastrampoline.so
     22480:	  trying file=x86_64/libblastrampoline.so
     22480:	  trying file=libblastrampoline.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libblastrampoline.so
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libblastrampoline.so
     22480:
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax__ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax___ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: isamax_64 (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_dbf16tod64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbdot64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbdtobf1664_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbf16tos64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbgemm64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbgemv64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: cblas_sbstobf1664_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: dbf16tod_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qamax_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qamin_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qasum_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qaxpy_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qcabs1_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qcopy_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qdot_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qgbmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qgemm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qgemv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qger_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qmax_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qmin_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qnrm2_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsbmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qscal_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qspmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qspr2_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qspr_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qswap_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsymm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsymv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyr2_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyr2k_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyr_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qsyrk_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtbmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtbsv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtpmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtpsv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrmm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrsm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qtrsv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxamax_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxamin_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxasum_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: qxnrm2_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbdot_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbdtobf16_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbf16tos_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbgemm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbgemv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: sbstobf16_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xaxpy_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xcopy_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xdotc_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xdotu_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgbmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgemm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgemv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgerc_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xgeru_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhbmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhemm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhemv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xher2_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xher2k_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xher_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xherk_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhpmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhpr2_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xhpr_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xqscal_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xscal_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xswap_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xsymm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xsyr2k_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xsyrk_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtbmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtbsv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtpmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtpsv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrmm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrmv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrsm_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: xtrsv_64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: MKL_Set_Num_Threads64_ (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so: error: symbol lookup error: undefined symbol: bli_thread_set_num_threads64_ (fatal)
     22480:	find library=libcholmod [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcholmod
     22480:	  trying file=tls/x86_64/libcholmod
     22480:	  trying file=tls/libcholmod
     22480:	  trying file=x86_64/libcholmod
     22480:	  trying file=libcholmod
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcholmod
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libcholmod
     22480:	  trying file=/lib64/libcholmod
     22480:	  trying file=/usr/lib64/tls/libcholmod
     22480:	  trying file=/usr/lib64/libcholmod
     22480:
     22480:	find library=libcholmod.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcholmod.so
     22480:	  trying file=tls/x86_64/libcholmod.so
     22480:	  trying file=tls/libcholmod.so
     22480:	  trying file=x86_64/libcholmod.so
     22480:	  trying file=libcholmod.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod.so
     22480:
     22480:	find library=libamd.so.2 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libamd.so.2
     22480:
     22480:	find library=libcolamd.so.2 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcolamd.so.2
     22480:
     22480:	find library=libsuitesparseconfig.so.5 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so.5
     22480:
     22480:	find library=libccolamd.so.2 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libccolamd.so.2
     22480:
     22480:	find library=libcamd.so.2 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcamd.so.2
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so.5
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcamd.so.2
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libccolamd.so.2
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcolamd.so.2
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libamd.so.2
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod.so
     22480:
     22480:	find library=libcholmod [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcholmod
     22480:	  trying file=tls/x86_64/libcholmod
     22480:	  trying file=tls/libcholmod
     22480:	  trying file=x86_64/libcholmod
     22480:	  trying file=libcholmod
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcholmod
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libcholmod
     22480:	  trying file=/lib64/libcholmod
     22480:	  trying file=/usr/lib64/tls/libcholmod
     22480:	  trying file=/usr/lib64/libcholmod
     22480:
     22480:	find library=libsuitesparseconfig [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libsuitesparseconfig
     22480:	  trying file=tls/x86_64/libsuitesparseconfig
     22480:	  trying file=tls/libsuitesparseconfig
     22480:	  trying file=x86_64/libsuitesparseconfig
     22480:	  trying file=libsuitesparseconfig
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libsuitesparseconfig
     22480:	 search cache=/etc/ld.so.cache
     22480:	 search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64		(system search path)
     22480:	  trying file=/lib64/tls/libsuitesparseconfig
     22480:	  trying file=/lib64/libsuitesparseconfig
     22480:	  trying file=/usr/lib64/tls/libsuitesparseconfig
     22480:	  trying file=/usr/lib64/libsuitesparseconfig
     22480:
     22480:	find library=libsuitesparseconfig.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libsuitesparseconfig.so
     22480:	  trying file=tls/x86_64/libsuitesparseconfig.so
     22480:	  trying file=tls/libsuitesparseconfig.so
     22480:	  trying file=x86_64/libsuitesparseconfig.so
     22480:	  trying file=libsuitesparseconfig.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so
     22480:
     22480:	find library=libnghttp2.so.14 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libnghttp2.so.14
     22480:	  trying file=tls/x86_64/libnghttp2.so.14
     22480:	  trying file=tls/libnghttp2.so.14
     22480:	  trying file=x86_64/libnghttp2.so.14
     22480:	  trying file=libnghttp2.so.14
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libnghttp2.so.14
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libnghttp2.so.14
     22480:
     22480:	find library=libcurl.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcurl.so
     22480:	  trying file=tls/x86_64/libcurl.so
     22480:	  trying file=tls/libcurl.so
     22480:	  trying file=x86_64/libcurl.so
     22480:	  trying file=libcurl.so
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcurl.so
     22480:
     22480:	find library=libssh2.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libssh2.so.1
     22480:	  trying file=tls/x86_64/libssh2.so.1
     22480:	  trying file=tls/libssh2.so.1
     22480:	  trying file=x86_64/libssh2.so.1
     22480:	  trying file=libssh2.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libssh2.so.1
     22480:
     22480:	find library=libmbedtls.so.14 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmbedtls.so.14
     22480:	  trying file=tls/x86_64/libmbedtls.so.14
     22480:	  trying file=tls/libmbedtls.so.14
     22480:	  trying file=x86_64/libmbedtls.so.14
     22480:	  trying file=libmbedtls.so.14
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmbedtls.so.14
     22480:
     22480:	find library=libmbedx509.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmbedx509.so.1
     22480:	  trying file=tls/x86_64/libmbedx509.so.1
     22480:	  trying file=tls/libmbedx509.so.1
     22480:	  trying file=x86_64/libmbedx509.so.1
     22480:	  trying file=libmbedx509.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmbedx509.so.1
     22480:
     22480:	find library=libmbedcrypto.so.7 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libmbedcrypto.so.7
     22480:	  trying file=tls/x86_64/libmbedcrypto.so.7
     22480:	  trying file=tls/libmbedcrypto.so.7
     22480:	  trying file=x86_64/libmbedcrypto.so.7
     22480:	  trying file=libmbedcrypto.so.7
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libmbedcrypto.so.7
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedcrypto.so.7
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedx509.so.1
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedtls.so.14
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libssh2.so.1
     22480:
     22480:
     22480:	calling init: /share/apps/julia/julia-1.9/beta3/lib/julia/libcurl.so
     22480:
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/JLLWrappers/7Zgw7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/LLVMExtra_jll/R9OeX_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Reexport/bTpYr_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/share/julia/compiled/v1.9/Statistics/ERcPL_F2yMK.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/Requires/IyxeS_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_fvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_sysimg_gvars_base (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	/home/feist/.julia/compiled/v1.9/CUDA_Driver_jll/QJyk7_Tpycd.so: error: symbol lookup error: undefined symbol: jl_pgcstack_func_slot (fatal)
     22480:	find library=libcuda.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22480:	  trying file=tls/x86_64/libcuda.so.1
     22480:	  trying file=tls/libcuda.so.1
     22480:	  trying file=x86_64/libcuda.so.1
     22480:	  trying file=libcuda.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libcuda.so.1
     22480:
     22480:
     22480:	calling init: /lib64/libcuda.so.1
     22480:
     22480:
     22480:	calling fini: /lib64/libcuda.so.1 [0]
     22480:
     22480:	find library=libcuda.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22480:	  trying file=tls/x86_64/libcuda.so.1
     22480:	  trying file=tls/libcuda.so.1
     22480:	  trying file=x86_64/libcuda.so.1
     22480:	  trying file=libcuda.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libcuda.so.1
     22480:
     22480:	find library=libcuda.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22480:	  trying file=tls/x86_64/libcuda.so.1
     22480:	  trying file=tls/libcuda.so.1
     22480:	  trying file=x86_64/libcuda.so.1
     22480:	  trying file=libcuda.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libcuda.so.1
     22480:
     22480:
     22480:	calling init: /lib64/libcuda.so.1
     22480:
     22480:
     22480:	calling fini: /lib64/libcuda.so.1 [0]
     22480:
     22480:	find library=libcuda.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22480:	  trying file=tls/x86_64/libcuda.so.1
     22480:	  trying file=tls/libcuda.so.1
     22480:	  trying file=x86_64/libcuda.so.1
     22480:	  trying file=libcuda.so.1
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib/julia:/share/apps/julia/julia-1.9/beta3/lib/julia/..		(RUNPATH from file /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/libcuda.so.1
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/julia/../libcuda.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libcuda.so.1
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/b5e755e06f4d49a5ab1a638eea5d75bf20c66e3d/lib/libcuda.so
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/b5e755e06f4d49a5ab1a638eea5d75bf20c66e3d/lib/libcuda.so [0]
     22480:
     22480:	find library=libcublasLt.so.11 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcublasLt.so.11
     22480:	  trying file=tls/x86_64/libcublasLt.so.11
     22480:	  trying file=tls/libcublasLt.so.11
     22480:	  trying file=x86_64/libcublasLt.so.11
     22480:	  trying file=libcublasLt.so.11
     22480:	 search path=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls/x86_64:/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls:/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/x86_64:/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib		(RUNPATH from file /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so)
     22480:	  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls/x86_64/libcublasLt.so.11
     22480:	  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls/libcublasLt.so.11
     22480:	  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/x86_64/libcublasLt.so.11
     22480:	  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublasLt.so.11
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublasLt.so.11
     22480:
     22480:	find library=libcuda.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcuda.so.1
     22480:	  trying file=tls/x86_64/libcuda.so.1
     22480:	  trying file=tls/libcuda.so.1
     22480:	  trying file=x86_64/libcuda.so.1
     22480:	  trying file=libcuda.so.1
     22480:	 search path=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib		(RUNPATH from file /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so)
     22480:	  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcuda.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libcuda.so.1
     22480:
     22480:
     22480:	calling init: /lib64/libcuda.so.1
     22480:
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx (fatal)
     22480:	find library=libnvrtc.so [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libnvrtc.so
     22480:	  trying file=tls/x86_64/libnvrtc.so
     22480:	  trying file=tls/libnvrtc.so
     22480:	  trying file=x86_64/libnvrtc.so
     22480:	  trying file=libnvrtc.so
     22480:	 search path=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib		(RUNPATH from file /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so)
     22480:	  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvrtc.so
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/usr/local/cuda-10.1/targets/x86_64-linux/lib/libnvrtc.so
     22480:
     22480:
     22480:	calling init: /usr/local/cuda-10.1/targets/x86_64-linux/lib/libnvrtc.so
     22480:
     22480:	/usr/local/cuda-10.1/targets/x86_64-linux/lib/libnvrtc.so: error: symbol lookup error: undefined symbol: __nvrtcCPEx (fatal)
     22480:	/usr/local/cuda-10.1/targets/x86_64-linux/lib/libnvrtc.so: error: symbol lookup error: undefined symbol: nvrtcGetCUBINSize (fatal)
     22480:	/usr/local/cuda-10.1/targets/x86_64-linux/lib/libnvrtc.so: error: symbol lookup error: undefined symbol: nvrtcGetCUBIN (fatal)
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcudart.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcufft.so
     22480:
     22480:	find library=libutil.so.1 [0]; searching
     22480:	 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
     22480:	  trying file=/share/apps/julia/julia-1.9/beta3/lib/libutil.so.1
     22480:	  trying file=tls/x86_64/libutil.so.1
     22480:	  trying file=tls/libutil.so.1
     22480:	  trying file=x86_64/libutil.so.1
     22480:	  trying file=libutil.so.1
     22480:	 search cache=/etc/ld.so.cache
     22480:	  trying file=/lib64/libutil.so.1
     22480:
     22480:
     22480:	calling init: /lib64/libutil.so.1
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcupti.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcurand.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolver.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolverMg.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusparse.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvToolsExt.so
     22480:
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvvm.so
     22480:
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuFlushGPUDirectRDMAWrites (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuModuleGetLoadingMode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuArrayGetMemoryRequirements (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuMipmappedArrayGetMemoryRequirements (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxPotentialClusterSize (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxActiveClusters (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemAllocNode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemAllocNodeGetParams (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemFreeNode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemFreeNodeGetParams (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGraphMemTrim (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGetGraphMemAttribute (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceSetGraphMemAttribute (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphInstantiateWithFlags (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2 (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphDebugDotPrint (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectCreate (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRetain (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRelease (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphRetainUserObject (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphReleaseUserObject (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeSetEnabled (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeGetEnabled (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:
     22480:	calling init: /home/feist/.julia/artifacts/9723dd10f25f310e5d2c3f1113ef158db3aa89b2/lib/libLLVMExtra-14.so
     22480:
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_type_to_llvm (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_symbol_name (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_symbol_name (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemmove (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemmove (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemhash_seed (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_options (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemcmp (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemchr (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iuv_pipe_open (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: iclose (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuFlushGPUDirectRDMAWrites (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuModuleGetLoadingMode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuArrayGetMemoryRequirements (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuMipmappedArrayGetMemoryRequirements (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxPotentialClusterSize (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxActiveClusters (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemAllocNode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemAllocNodeGetParams (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemFreeNode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemFreeNodeGetParams (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGraphMemTrim (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGetGraphMemAttribute (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceSetGraphMemAttribute (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphInstantiateWithFlags (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2 (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphDebugDotPrint (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectCreate (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRetain (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRelease (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphRetainUserObject (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphReleaseUserObject (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeSetEnabled (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeGetEnabled (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGetProcAddress (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuFlushGPUDirectRDMAWrites (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuModuleGetLoadingMode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuArrayGetMemoryRequirements (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuMipmappedArrayGetMemoryRequirements (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuLaunchKernelEx_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxPotentialClusterSize (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuOccupancyMaxActiveClusters (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemAllocNode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemAllocNodeGetParams (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphAddMemFreeNode (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphMemFreeNodeGetParams (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGraphMemTrim (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceGetGraphMemAttribute (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuDeviceSetGraphMemAttribute (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphInstantiateWithFlags (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2 (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamGetCaptureInfo_v2_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuStreamUpdateCaptureDependencies_ptsz (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphDebugDotPrint (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectCreate (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRetain (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuUserObjectRelease (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphRetainUserObject (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphReleaseUserObject (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeSetEnabled (fatal)
     22480:	/lib64/libcuda.so.1: error: symbol lookup error: undefined symbol: cuGraphNodeGetEnabled (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
1.0
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: imemset (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_stdout (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_stdout (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_putb (fatal)
     22480:	/share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1: error: symbol lookup error: undefined symbol: ijl_uv_putb (fatal)
     22480:
     22480:	calling fini: /usr/local/cuda-10.1/targets/x86_64-linux/lib/libnvrtc.so [0]
     22480:
     22480:
     22480:	calling fini: julia [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenlibm.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-codegen.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/sys.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libpcre2-8.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmpfr.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgmp.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgomp.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libssp.so.0 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libopenblas64_.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgfortran.so.5 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libquadmath.so.0 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcholmod.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libblastrampoline.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libamd.so.2 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcolamd.so.2 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libccolamd.so.2 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcamd.so.2 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libsuitesparseconfig.so.5 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libcurl.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libnghttp2.so.14 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libssh2.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedtls.so.14 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedx509.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libmbedcrypto.so.7 [0]
     22480:
     22480:
     22480:	calling fini: /lib64/libcuda.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcudart.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcufft.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcupti.so [0]
     22480:
     22480:
     22480:	calling fini: /lib64/libutil.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcurand.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolver.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusolverMg.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublasLt.so.11 [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcusparse.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvToolsExt.so [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libnvvm.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libjulia-internal.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libatomic.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/libjulia.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /home/feist/.julia/artifacts/9723dd10f25f310e5d2c3f1113ef158db3aa89b2/lib/libLLVMExtra-14.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libLLVM-14jl.so [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libz.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /lib64/librt.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libstdc++.so.6 [0]
     22480:
     22480:
     22480:	calling fini: /share/apps/julia/julia-1.9/beta3/lib/julia/libgcc_s.so.1 [0]
     22480:
     22480:
     22480:	calling fini: /lib64/libm.so.6 [0]
     22480:
     22480:
     22480:	calling fini: /lib64/libpthread.so.0 [0]
     22480:
     22480:
     22480:	calling fini: /lib64/libdl.so.2 [0]
     22480:

@maleadt
Copy link
Member

maleadt commented Feb 6, 2023

Thanks. That shows the issue happens with libcublasLt, a dependency of libcublas:

find library=libcublasLt.so.11 [0]; searching
 search path=/share/apps/julia/julia-1.9/beta3/lib:/usr/local/cuda/lib64:/usr/local/cuda-11.2/lib64:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcublasLt.so.11
  trying file=/usr/local/cuda/lib64/libcublasLt.so.11
calling init: /usr/local/cuda/lib64/libcublasLt.so.11

vs

find library=libcublasLt.so.11 [0]; searching
 search path=/share/apps/julia/julia-1.9/beta3/lib:tls/x86_64:tls:x86_64:		(LD_LIBRARY_PATH)
  trying file=/share/apps/julia/julia-1.9/beta3/lib/libcublasLt.so.11
  trying file=tls/x86_64/libcublasLt.so.11
  trying file=tls/libcublasLt.so.11
  trying file=x86_64/libcublasLt.so.11
  trying file=libcublasLt.so.11
 search path=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls/x86_64:/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls:/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/x86_64:/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib		(RUNPATH from file /home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublas.so)
  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls/x86_64/libcublasLt.so.11
  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/tls/libcublasLt.so.11
  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/x86_64/libcublasLt.so.11
  trying file=/home/feist/.julia/artifacts/534759902986a160c0f96bffe4d7af29a256295c/lib/libcublasLt.so.11

And through RUNPATH is set to $ORIGIN, LD_LIBRARY_PATH takes precedence.

Maybe we should be patching RPATH=$ORIGIN in? Alternatively, we could add a LibraryProduct for libcublasLt to the JLL, but then we probably need to do that for every CUDA library (for example, libcudnn depends on libcudnn_(adv_infer|adv_train|cnn_infer|cnn_train|ops_infer|ops_train), but those sublibraries aren't meant for public consumption so I'd rather not add each of those to the JLL).

The other position is to deem setting LD_LIBRARY_PATH to be a user error.

cc @vchuravy @staticfloat

@staticfloat
Copy link
Contributor

First, I consider LD_LIBRARY_PATH to be a kind of "warranty-voiding" action. If that is set, I think it's reasonable for Julia to misbehave every now and then. (The only more disruptive environment variable, IMO, is LD_PRELOAD ;) ). That being said, JLLs do their best to be resistant to that by opening libraries by full path, in dependency order (as much as possible), which dodges most of these problems. So adding all those libraries as explicit products is probably the solution I would advocate.

Note that setting up an RPATH isn't necessarily a solution either, because LD_LIBRARY_PATH is searched before the DT_RUNPATH tags, which are the default ELF tags used by modern ld when linking. Older ld uses DT_RPATH, which is searched first, but usage of DT_RPATH is discouraged precisely because it is not overridden by LD_LIBRARY_PATH which is considered a bug, because if the user has set LD_LIBRARY_PATH then they supposedly really know what they're doing and you should not ignore them.

patchelf generates the new-style DT_RUNPATH tags, and so would not fix this issue either.

So in the end, my advice is:

  1. Don't set LD_LIBRARY_PATH on a regular basis. Only use it for one-off debugging.
  2. Create library products for everything you want your users to typically load.

@maleadt
Copy link
Member

maleadt commented Feb 7, 2023

First, I consider LD_LIBRARY_PATH to be a kind of "warranty-voiding" action. If that is set, I think it's reasonable for Julia to misbehave every now and then.

I agree, but tell that to Nix or HPC users 🙂

Thanks for the details on RPATH, I'll just add LibraryProducts for now (see JuliaPackaging/Yggdrasil#6217).

JLLs do their best to be resistant to that by opening libraries by full path, in dependency order (as much as possible)

Some libraries don't expose their dependencies though. Take libcudnn.so:

	linux-vdso.so.1 (0x00007ffff3d54000)
	librt.so.1 => /usr/lib/librt.so.1 (0x00007f333a258000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f333a253000)
	libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f333a24e000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f3339c00000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f3339f18000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f333a22c000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f3339a19000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f333a274000)

Yet:

julia> cudnnActivationDescriptor(CUDNN_ACTIVATION_RELU,CUDNN_NOT_PROPAGATE_NAN,0)
    550427:	find library=libcudnn_ops_infer.so.8 [0]; searching
    550427:	 search path=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/glibc-hwcaps/x86-64-v3:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/glibc-hwcaps/x86-64-v2:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls/x86_64/x86_64:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls/x86_64:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls/x86_64:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/x86_64/x86_64:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/x86_64:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/x86_64:/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib		(RUNPATH from file /home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/libcudnn.so)
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/glibc-hwcaps/x86-64-v3/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/glibc-hwcaps/x86-64-v2/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls/x86_64/x86_64/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls/x86_64/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls/x86_64/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/tls/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/x86_64/x86_64/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/x86_64/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/x86_64/libcudnn_ops_infer.so.8
    550427:	  trying file=/home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/libcudnn_ops_infer.so.8
    550427:
    550427:
    550427:	calling init: /home/tim/Julia/depot/artifacts/39edbd07a46d182c2681130c16ff339251297514/lib/libcudnn_ops_infer.so.8
julia>

i.e. the library uses a plug-in system that dynamically dlopens sublibraries by itself. And this isn't uncommon, openCL works like too, as does the oneAPI ecosystem. Adding these private sublibraries to the list of LibraryProducts (which is ugly) then may not help, depending on how they are loaded.

@staticfloat
Copy link
Contributor

i.e. the library uses a plug-in system that dynamically dlopens sublibraries by itself.

If the library dlopen's by calculating a path relative to itself, that will work just fine. If it loads it by SONAME, that will also be just fine, as we'll already have loaded something with the same SONAME. The only case in which it won't work is if it tries to load the library by a non-canonical name; e.g. something like loading libcudnn_ops_infer.so; that will cause the dynamic linker to do a disk search, and it won't find something with that SONAME previously loaded, nor will it find it on disk due to the issues previously mentioned. Luckily, in the Linux world, most tools try hard to encourage you to use the SONAME when you load things, for exactly this reason.

@maleadt
Copy link
Member

maleadt commented Feb 7, 2023

If it loads it by SONAME, that will also be just fine, as we'll already have loaded something with the same SONAME.

But how do you we will have loaded the sublibrary? How does the JLL know to load the sublibrary first if there's no dependency path from libcudnn.so to libcudnn_ops_infer.so?

EDIT: in this case it will, but I think some OpenCL libraries dlopen the implementation sublibrary during initialization, i.e., not lazily when a call happens. Maybe I shouldn't worry about this though until people complain 🙂

@staticfloat
Copy link
Contributor

Yeah, we don't have a good way right now to force an ordering on products within a single JLL.

@maleadt
Copy link
Member

maleadt commented Feb 7, 2023

Anyway, this is fixed for the specific issue at hand here. It will take a while before this lands in a release though, so please continue not populating LD_LIBRARY_PATH with a toolkit directory for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants