Precompilation segfault after SnoopPrecompile was updated from v1.0.1 to v1.0.3

[lanceXwq]

I am running into this problem with Julia 1.8.5, Makie v0.19.1, and Ubuntu 22.04.01. This issue was not seen on my Apple silicon Mac.

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

julia> using GLMakie
[ Info: Precompiling GLMakie [e9467ef8-e4e7-5192-8a1a-b1aee30e663a]

signal (11): Segmentation fault
in expression starting at /home/lancexwq/.julia/packages/GLMakie/6Blp9/src/GLMakie.jl:10
jl_ptr_offset at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1152 [inlined]
jl_deserialize_struct at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:2084 [inlined]
jl_deserialize_value_any at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:2186 [inlined]
jl_deserialize_value at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:2319
jl_deserialize_value_code_instance at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:1974 [inlined]
jl_deserialize_value at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:2265
jl_deserialize_value_array at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:1777
jl_deserialize_value at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:2230
_jl_restore_incremental at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:3364
ijl_restore_incremental at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/dump.c:3436
_include_from_serialized at ./loading.jl:807
_require_search_from_serialized at ./loading.jl:1039
_require at ./loading.jl:1315
_require_prelocked at ./loading.jl:1200
macro expansion at ./loading.jl:1180 [inlined]
macro expansion at ./lock.jl:223 [inlined]
require at ./loading.jl:1144
jfptr_require_36652.clone_1 at /home/lancexwq/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
call_require at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:462 [inlined]
eval_import_path at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:499
jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:725
jl_eval_module_expr at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:203 [inlined]
jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:709
jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
_include at ./loading.jl:1488
include at ./Base.jl:419 [inlined]
include_package_for_output at ./loading.jl:1554
jfptr_include_package_for_output_39098.clone_1 at /home/lancexwq/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
do_call at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:215
eval_stmt_value at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:166 [inlined]
eval_body at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:612
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:750
jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:906
jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
include_string at ./loading.jl:1428
include_string at ./loading.jl:1438
_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
exec_options at ./client.jl:301
_start at ./client.jl:522
jfptr__start_38041.clone_1 at /home/lancexwq/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
true_main at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/jlapi.c:575
jl_repl_entrypoint at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/jlapi.c:719
main at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/cli/loader_exe.c:59
unknown function (ip: 0x7f8585b56d8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 13721462 (Pool: 13717199; Big: 4263); GC: 9
ERROR: Failed to precompile GLMakie [e9467ef8-e4e7-5192-8a1a-b1aee30e663a] to /home/lancexwq/.julia/compiled/v1.8/GLMakie/jl_WVGEdy.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1707
 [3] compilecache
   @ ./loading.jl:1651 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144

[timholy]

I can't replicate this locally.

It's certainly possible that SnoopPrecompile 1.0.3 triggered this error, but it would be a little surprising since the diff between the two versions is just https://github.com/timholy/SnoopCompile.jl/compare/c2f8728275c785f802068fa3d267c916781f94b9..fa775b3081ddc8f66aa601a35752daeced39ec8b#diff-6765d7ec1aae3a2e04f5acefbc3a91e6266c8af453f1cd3f44c9b85ff14e6ba3 (just the src/SnoopPrecompile.jl file).

Maybe clean out your .julia/v1.8/compiled/* folder and try again? It's possible your repos have gotten into an inconsistent state, especially if you've used any ENV paths to control precompilation. The Preferences.jl mechanism in SnoopPrecompile 1.0.3 is the "correct" way to handle user precompilation preferences.

[lanceXwq]

Cleaning out .julia/compiled/v1.8/* does resolve this issue. Thank you!

[SimonDanisch]

It does seem to happen on the ci though:
https://github.com/MakieOrg/Makie.jl/actions/runs/3903680309/jobs/6668379893
Maybe the compile cache isn't correctly invalidated when something in the precompile changes?

[t-bltg]

Thinking out loud, PRECOMPILATION_CACHE_PATH seems empty thus julia-actions/cache should not cache ~/.julia/compiled but out of safety maybe cleaning up the actions caches in https://github.com/MakieOrg/Makie.jl/actions/caches might help ?

I had to do that recently in Plots.

See also julia-actions/cache#42.

[SimonDanisch]

but... shouldn't julia make sure that you don't need to manually clean the precompile cache? On the CI we want to test that no user runs into segfaults, so I think the CI shouldn't do anything like that...
If we debug the issue and it turns out that it's a julia fault that's hard to fix and it turns up in our CI all the time, we can think about adding a warning to the docs + purge the folder in the CI, but otherwise I wouldn't like to resolve to this.

[timholy]

Does your precompilation pipeline involve any ENV variables? If so, delete that code and use Preferences.jl instead. ENV variables are not logged as dependent state in the cache files, so making any aspect of compilation dependent on them is dangerous.

Note this only affects ENV variables that get accessed at compile-time and whose value would lead to different compiled code. Runtime ENV variables are perfectly fine.

[timholy]

In particular, if Makie itself is switching off SnoopPrecompile workloads using an ENV variable, please see https://discourse.julialang.org/t/psa-for-snoopprecompile-turning-off-extra-workload-for-specific-packages/92865

[jkrumbiegel]

Yes I've seen the buildup to that and will not go further with the ENV var PR for Makie

[timholy]

Just searching the repo for ENV:

• does this ever get hardcoded in, or is it always accessed at runtime?

  Makie.jl/docs/documentation/faq.md

  Line 86 in e85847b

  - `ENV["FREETYPE_ABSTRACTION_FONT_PATH"] = "/path/to/your/fonts"`

  (again, runtime is perfectly fine, but const fontpath = ENV["FREETYPE_ABSTRACTION_FONT_PATH"] would be problematic because it won't update if the user changes the ENV variable)

• Makie.jl/metrics/collect_metrics.jl

  Line 10 in e85847b

  metric_target_raw = get(ENV, "METRIC_TARGET", "")

  seems problematic if it gets precompiled in

That's all I found, so it's not much. Doesn't seem likely to be the culprit 🤔

[jkrumbiegel]

The first one would be FreeTypeAbstraction, and I think it's only for runtime path lookup, but might warrant a check. The second one is not a Makie source file, but a script for collecting metrics on CI :)