Segmentation fault in Julia 1.7 only

[schlichtanders]

Hi all,

I posted this also on discourse.

The test of a new package of mine ExtensibleEffects are working well on Julia 1.6, however crash with a Segmentation fault in Julia nightly, i.e. Julia 1.7. I guess this might be important.

By now the package is also registered, hence you can easily reproduce the test by running

julia> ]
pkg> add ExtensibleEffects
pkg> test ExtensibleEffects

Doing this on nightly should reproduce the segmentation fault.

Click to expand! (Segmentation Fault Error log)

signal (11): Segmentation fault
in expression starting at /home/runner/work/ExtensibleEffects.jl/ExtensibleEffects.jl/test/VectorIterable.jl:17
gc_mark_loop at /buildworker/worker/package_linux64/build/src/gc.c:2523
_jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:3028
jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:3236
maybe_collect at /buildworker/worker/package_linux64/build/src/gc.c:881 [inlined]
jl_gc_pool_alloc at /buildworker/worker/package_linux64/build/src/gc.c:1205
iterate at ./compiler/ssair/ir.jl:1314
batch_inline! at ./compiler/ssair/inlining.jl:592
ssa_inlining_pass! at ./compiler/ssair/inlining.jl:75
jfptr_ssa_inlining_passNOT._9395.clone_1 at /opt/hostedtoolcache/julia/nightly/x64/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2245 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2427
run_passes at ./compiler/optimize.jl:299
optimize at ./compiler/optimize.jl:288 [inlined]
_typeinf at ./compiler/typeinfer.jl:255
typeinf at ./compiler/typeinfer.jl:209
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:520
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:168
abstract_call_known at ./compiler/abstractinterpretation.jl:1270
abstract_call at ./compiler/abstractinterpretation.jl:1324
abstract_call at ./compiler/abstractinterpretation.jl:1309
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1463
typeinf_local at ./compiler/abstractinterpretation.jl:1833
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1940
_typeinf at ./compiler/typeinfer.jl:226
typeinf at ./compiler/typeinfer.jl:209
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:520
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:168
abstract_call_known at ./compiler/abstractinterpretation.jl:1270
abstract_call at ./compiler/abstractinterpretation.jl:1324
abstract_call at ./compiler/abstractinterpretation.jl:1309
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1463
typeinf_local at ./compiler/abstractinterpretation.jl:1833
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1940
_typeinf at ./compiler/typeinfer.jl:226
typeinf at ./compiler/typeinfer.jl:209
typeinf_ext at ./compiler/typeinfer.jl:908
typeinf_ext_toplevel at ./compiler/typeinfer.jl:941
typeinf_ext_toplevel at ./compiler/typeinfer.jl:937
jfptr_typeinf_ext_toplevel_9381.clone_1 at /opt/hostedtoolcache/julia/nightly/x64/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2245 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2427
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1787 [inlined]
jl_type_infer at /buildworker/worker/package_linux64/build/src/gf.c:295
jl_generate_fptr at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:338
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2245 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2427
_include at ./loading.jl:1253
include at ./client.jl:451
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2245 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2427
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1787 [inlined]
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:125
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:214
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:165 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:583
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:727
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:885
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:830
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:944
eval at ./boot.jl:373 [inlined]
exec_options at ./client.jl:268
_start at ./client.jl:495
jfptr__start_20685.clone_1 at /opt/hostedtoolcache/julia/nightly/x64/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2245 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2427
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1787 [inlined]
true_main at /buildworker/worker/package_linux64/build/src/jlapi.c:559
jl_repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:701
main at /buildworker/worker/package_linux64/build/cli/loader_exe.c:42
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /opt/hostedtoolcache/julia/nightly/x64/bin/julia (unknown line)
Allocations: 21183232 (Pool: 21171191; Big: 12041); GC: 23
ERROR: Package ExtensibleEffects errored during testing (received signal: 11)
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.8/Pkg/src/Types.jl:68
 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
   @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.8/Pkg/src/Operations.jl:1681
 [3] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::Base.Pairs{Symbol, IOContext{Base.PipeEndpoint}, Tuple{Symbol}, NamedTuple{(:io,), Tuple{IOContext{Base.PipeEndpoint}}}})
   @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:426
 [4] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{Base.PipeEndpoint}, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol, Symbol}, NamedTuple{(:coverage, :force_latest_compatible_version), Tuple{Bool, Bool}}})
   @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:154
 [5] test(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol, Symbol}, NamedTuple{(:coverage, :force_latest_compatible_version), Tuple{Bool, Bool}}})
   @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:169
 [6] top-level scope
   @ none:1
Vector/Iterable: 
Error: Process completed with exit code 1.

[lukego]

I bisected this as far as line 282 on commit cf634a8:

rettype = Conditional(SlotNumber(slot), vtype, elsetype) # record a Conditional improvement to this slot

Since the commit is in the compiler and the crash is in GC this optimization is presumably generating code that corrupts the heap.

If the type information backpropagated from boolean expressions here is wrong for some reason then should the symptom be bad codegen or rather a compiletime/runtime type error? That might give a hint about whether the problem is in this code that constructs the Conditional verses a later part of the compiler that uses it.

[maleadt]

I had creduce work on this for a while, and came up with the following MWE:

struct b end
struct f{g, d}
  effectful::g
  function f(effectful::h, ::d=b) where {h, d}
      new{h, d}(effectful)
  end
end
effect(k) = f(k)
w(::Tuple{}, eff) = eff
w(m, eff) = o(first(m), w(Base.tail(m), eff))
o(handler, eff) = p(handler, eff.effectful) || f(eff.effectful)
p(handler, k) = false
macro u(block)
 Expr(:call, effect, block.args)
end
p(::Type{h}, ::h) where h<:AbstractArray = true
v = @u begin end
w((b, Vector), v)

signal (11): Segmentation fault
in expression starting at /home/tim/Julia/tools/creduce/main.jl:18
typekeyvalue_hash at /buildworker/worker/package_linux64/build/src/jltypes.c:1135 [inlined]
lookup_typevalue at /buildworker/worker/package_linux64/build/src/jltypes.c:720
jl_inst_arg_tuple_type at /buildworker/worker/package_linux64/build/src/jltypes.c:1558
arg_type_tuple at /buildworker/worker/package_linux64/build/src/gf.c:1843 [inlined]
jl_lookup_generic_ at /buildworker/worker/package_linux64/build/src/gf.c:2371 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2423
o at /home/tim/Julia/tools/creduce/main.jl:11

Reproduces on 1.7 and nightly.

I only looked for segmentation faults, and the backtrace here is different than the originally-reported one... Maybe it is related, though?

[vtjnash]

That is helpful. In Cthulhu, we see that reaches this subtyping bug (which we started to get wrong in v1.2):

julia> typeintersect(Tuple{DataType, Vector{Any}}, Tuple{Type{T}, T} where T<:AbstractArray, )
Tuple{Type{Vector{Any}}, Vector{Any}}

julia> ans == Tuple{Type{T}, Vector{Any}} where Vector{Any}<:T<:AbstractArray
false

[JeffBezanson]

Ok I believe I've determined that that intersection bug was introduced by 57a3b08.