Package extension loading issues in stacked environments #55033
Description
julia> versioninfo()
Julia Version 1.12.0-DEV.814
Commit c3883824b00 (2024-07-04 16:05 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 16 × AMD Ryzen 7 7800X3D 8-Core Processor
WORD_SIZE: 64
LLVM: libLLVM-17.0.6 (ORCJIT, znver4)
Threads: 1 default, 0 interactive, 1 GC (on 16 virtual cores)
Environment:
JULIA_DEPOT_PATH = /tmp/new-depot
Here is a MWE:
First set up a stacked environment where the global environment has an older version of TranscodingStreams
JULIA_DEPOT_PATH=/tmp/new-depot julia +nightly -e '
using Pkg
pkg"add TranscodingStreams@0.10.8"
pkg"activate @stacked"; pkg"add TranscodingStreams@0.10.9 Test Random"
'Then run:
JULIA_DEBUG=loading JULIA_DEPOT_PATH=/tmp/new-depot julia +nightly -e '
using Pkg
using TranscodingStreams
pkg"activate @stacked"; using Test, Random
'This results in the error:
ERROR: LoadError: UndefVarError: `test_roundtrip_seekstart` not defined in `TranscodingStreams`
test_roundtrip_seekstart was added in TranscodingStreams v0.10.9. It looks like v0.10.9 of the TranscodingStreams TestExt extension is being used together with TranscodingStreams v0.10.8.
Full error
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Printf/3FQLY_ZPBod.so for Printf [de0858da-6303-5e67-8744-51eddeeeb8d7]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Dates/p8See_ZPBod.so for Dates [ade2ca70-3891-5945-98fb-dc099432e06a]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/TOML/mjrwE_ZPBod.so for TOML [fa267f1f-6049-4f14-aa54-33bafae1ed76]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/NetworkOptions/J8H6s_ZPBod.so for NetworkOptions [ca575930-c2e3-43a9-ace4-1e988b2c1908]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/MbedTLS_jll/u5NEn_ZPBod.so for MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/LibSSH2_jll/K6mup_ZPBod.so for LibSSH2_jll [29816b5a-b9ab-546f-933c-edad1886dfa8]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/LibGit2_jll/nfCpg_ZPBod.so for LibGit2_jll [e37daf67-58a4-590a-8e99-b0245dd2ffc5]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/LibGit2/xrYJZ_ZPBod.so for LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/ArgTools/aGHFV_ZPBod.so for ArgTools [0dad84c5-d112-42e6-8d28-ef12dabb789f]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/nghttp2_jll/KTGSA_ZPBod.so for nghttp2_jll [8e850ede-7688-5339-a07c-302acd2aaf8d]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/LibCURL_jll/9JWaY_ZPBod.so for LibCURL_jll [deac9b47-8bc7-5906-a0fe-35ac56dc84c0]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/MozillaCACerts_jll/XKIUi_ZPBod.so for MozillaCACerts_jll [14a3606d-f60d-562e-9121-12d972cd8159]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/LibCURL/ht49g_ZPBod.so for LibCURL [b27032c2-a3e7-50c8-80cd-2d36dbcbfd21]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Downloads/eiA4B_ZPBod.so for Downloads [f43a241f-c20a-4ad4-852c-f6b1247861c6]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Tar/G9ZYP_ZPBod.so for Tar [a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/p7zip_jll/dfuGM_ZPBod.so for p7zip_jll [3f19e933-33d8-53b3-aaab-bd5110c3b7a0]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/UUIDs/SIw1t_ZPBod.so for UUIDs [cf7118a7-6976-5b1a-9a39-7adc72f591a4]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Logging/PWFjL_ZPBod.so for Logging [56ddb016-857b-54e1-b83d-db4d58db5568]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Base64/D7K0n_ZPBod.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/StyledStrings/UcVoM_ZPBod.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/JuliaSyntaxHighlighting/lOBbc_ZPBod.so for JuliaSyntaxHighlighting [dc6e5ff7-fb65-4e79-a425-ec3bc9c03011]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Markdown/AREjX_ZPBod.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Pkg/tUTdb_ZPBod.so for Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/TranscodingStreams/eJQ7D_ZPBod.so for TranscodingStreams [3bb67fe8-82b1-5028-8e26-92a6c54297fa]
└ @ Base loading.jl:1227
┌ Warning: The Pkg REPL mode is intended for interactive use only, and should not be used from scripts. It is recommended to use the functional API instead.
└ @ Pkg.REPLMode ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/Pkg/src/REPLMode/REPLMode.jl:388
Activating project at `/tmp/new-depot/environments/stacked`
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/InteractiveUtils/0TrXF_j8m9s.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Serialization/zGad9_j8m9s.so for Serialization [9e88b42a-f829-5b0c-bbe9-9e923198166b]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Test/JfdTE_j8m9s.so for Test [8dfed614-e22c-5e08-85e1-65c5234f0b40]
└ @ Base loading.jl:1227
┌ Debug: Rejecting cache file /tmp/new-depot/compiled/v1.12/TestExt/puxfO_j8m9s.ji because module Base.PkgId(Base.UUID("3bb67fe8-82b1-5028-8e26-92a6c54297fa"), "TranscodingStreams") is already loaded and incompatible.
└ @ Base loading.jl:3754
┌ Debug: Rejecting cache file /tmp/new-depot/compiled/v1.12/TranscodingStreams/eJQ7D_ZPBod.ji because it is for file /tmp/new-depot/packages/TranscodingStreams/o7roJ/src/TranscodingStreams.jl not file /tmp/new-depot/packages/TranscodingStreams/F7ot4/src/TranscodingStreams.jl
└ @ Base loading.jl:3792
┌ Debug: Rejecting cache file /tmp/new-depot/compiled/v1.12/TestExt/puxfO_j8m9s.ji because module Base.PkgId(Base.UUID("3bb67fe8-82b1-5028-8e26-92a6c54297fa"), "TranscodingStreams") is already loaded and incompatible.
└ @ Base loading.jl:3754
┌ Debug: Rejecting cache file /tmp/new-depot/compiled/v1.12/TestExt/puxfO_j8m9s.ji because module Base.PkgId(Base.UUID("3bb67fe8-82b1-5028-8e26-92a6c54297fa"), "TranscodingStreams") is already loaded and incompatible.
└ @ Base loading.jl:3754
┌ Debug: Precompiling TestExt [29286dec-c893-52d1-88fe-e07e79eb4405] (cache misses: wrong dep version loaded (2))
└ @ Base loading.jl:2965
┌ Debug: Generating object cache file for TestExt [29286dec-c893-52d1-88fe-e07e79eb4405]
└ @ Base loading.jl:2853
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Base64/D7K0n_ZPBod.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/StyledStrings/UcVoM_ZPBod.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/JuliaSyntaxHighlighting/lOBbc_ZPBod.so for JuliaSyntaxHighlighting [dc6e5ff7-fb65-4e79-a425-ec3bc9c03011]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Markdown/AREjX_ZPBod.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/InteractiveUtils/0TrXF_j8m9s.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Serialization/zGad9_j8m9s.so for Serialization [9e88b42a-f829-5b0c-bbe9-9e923198166b]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Logging/PWFjL_ZPBod.so for Logging [56ddb016-857b-54e1-b83d-db4d58db5568]
└ @ Base loading.jl:1227
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/Test/JfdTE_j8m9s.so for Test [8dfed614-e22c-5e08-85e1-65c5234f0b40]
└ @ Base loading.jl:1227
┌ Debug: Rejecting cache file /tmp/new-depot/compiled/v1.12/TranscodingStreams/eJQ7D_j8m9s.ji because it provides the wrong build_id (got fafbfcfd-5107-b5f0-0000-d8b3946d8ced) for Base.PkgId(Base.UUID("3bb67fe8-82b1-5028-8e26-92a6c54297fa"), "TranscodingStreams") (want fafbfcfd-daad-5a53-0000-d8b369de787b)
└ @ Base loading.jl:3782
┌ Debug: Loading object cache file /tmp/new-depot/compiled/v1.12/TranscodingStreams/eJQ7D_ZPBod.so for TranscodingStreams [3bb67fe8-82b1-5028-8e26-92a6c54297fa]
└ @ Base loading.jl:1227
ERROR: LoadError: UndefVarError: `test_roundtrip_seekstart` not defined in `TranscodingStreams`
Stacktrace:
[1] getproperty(x::Module, f::Symbol)
@ Base ./Base.jl:42
[2] top-level scope
@ /tmp/new-depot/packages/TranscodingStreams/F7ot4/ext/TestExt.jl:65
[3] include
@ ./Base.jl:580 [inlined]
[4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
@ Base ./loading.jl:2797
[5] top-level scope
@ stdin:4
[6] eval
@ ./boot.jl:438 [inlined]
[7] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
@ Base ./loading.jl:2619
[8] include_string
@ ./loading.jl:2629 [inlined]
[9] exec_options(opts::Base.JLOptions)
@ Base ./client.jl:322
[10] _start()
@ Base ./client.jl:532
in expression starting at /tmp/new-depot/packages/TranscodingStreams/F7ot4/ext/TestExt.jl:1
in expression starting at stdin:4
┌ Error: Error during loading of extension TestExt of TranscodingStreams, use `Base.retry_load_extensions()` to retry.
│ exception =
│ 1-element ExceptionStack:
│ Failed to precompile TestExt [29286dec-c893-52d1-88fe-e07e79eb4405] to "/tmp/new-depot/compiled/v1.12/TestExt/jl_zzTyEj".
│ Stacktrace:
│ [1] error(s::String)
│ @ Base ./error.jl:44
│ [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool; flags::Cmd, cacheflags::Base.CacheFlags, reasons::Dict{String, Int64})
│ @ Base ./loading.jl:3064
│ [3] (::Base.var"#1100#1101"{Base.PkgId})()
│ @ Base ./loading.jl:2464
│ [4] mkpidlock(f::Base.var"#1100#1101"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
│ @ FileWatching.Pidfile ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:93
│ [5] #mkpidlock#6
│ @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:88 [inlined]
│ [6] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
│ @ FileWatching.Pidfile ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:114
│ [7] #invokelatest#2
│ @ ./essentials.jl:1047 [inlined]
│ [8] invokelatest
│ @ ./essentials.jl:1042 [inlined]
│ [9] maybe_cachefile_lock(f::Base.var"#1100#1101"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
│ @ Base ./loading.jl:3629
│ [10] maybe_cachefile_lock
│ @ ./loading.jl:3626 [inlined]
│ [11] _require(pkg::Base.PkgId, env::Nothing)
│ @ Base ./loading.jl:2460
│ [12] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│ @ Base ./loading.jl:2285
│ [13] #invoke_in_world#3
│ @ ./essentials.jl:1079 [inlined]
│ [14] invoke_in_world
│ @ ./essentials.jl:1076 [inlined]
│ [15] _require_prelocked
│ @ ./loading.jl:2276 [inlined]
│ [16] _require_prelocked
│ @ ./loading.jl:2275 [inlined]
│ [17] run_extension_callbacks(extid::Base.ExtensionId)
│ @ Base ./loading.jl:1479
│ [18] run_extension_callbacks(pkgid::Base.PkgId)
│ @ Base ./loading.jl:1514
│ [19] run_package_callbacks(modkey::Base.PkgId)
│ @ Base ./loading.jl:1335
│ [20] __require_prelocked(uuidkey::Base.PkgId, env::String)
│ @ Base ./loading.jl:2297
│ [21] #invoke_in_world#3
│ @ ./essentials.jl:1079 [inlined]
│ [22] invoke_in_world
│ @ ./essentials.jl:1076 [inlined]
│ [23] _require_prelocked(uuidkey::Base.PkgId, env::String)
│ @ Base ./loading.jl:2276
│ [24] macro expansion
│ @ ./loading.jl:2215 [inlined]
│ [25] macro expansion
│ @ ./lock.jl:273 [inlined]
│ [26] __require(into::Module, mod::Symbol)
│ @ Base ./loading.jl:2172
│ [27] #invoke_in_world#3
│ @ ./essentials.jl:1079 [inlined]
│ [28] invoke_in_world
│ @ ./essentials.jl:1076 [inlined]
│ [29] require(into::Module, mod::Symbol)
│ @ Base ./loading.jl:2165
│ [30] eval
│ @ ./boot.jl:438 [inlined]
│ [31] exec_options(opts::Base.JLOptions)
│ @ Base ./client.jl:290
│ [32] _start()
│ @ Base ./client.jl:532
└ @ Base loading.jl:1485