invalidation_trees() throws error: Collection has multiple elements

[ConnectedSystems]

Hello!

I'm developing a package (http://github.com/open-AIMS/ADRIA.jl) which has grown quite large. Initial precompilation can take a long time: typically 2mins but I've seen upwards of 5mins as well.

I was hoping to use SnoopCompile to help identify where the biggest causes are but following the tutorial the invalidation_trees() step throws an error: ERROR: ArgumentError: Collection has multiple elements, must contain exactly 1 element.

Is this a bug in SnoopCompile or am I doing something wrong?

julia> versioninfo()
Julia Version 1.8.3
Commit 0434deb161 (2022-11-14 20:14 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 16 × AMD Ryzen 7 6800HS Creator Edition
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver3)
  Threads: 8 on 16 virtual cores
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 8

julia> @time using ADRIA
[ Info: Precompiling ADRIA [7dc409a7-fbe5-4c9d-b3e2-b0c19a6ba600]
139.955745 seconds (96.93 M allocations: 6.197 GiB, 2.07% gc time, 4.18% compilation time: 48% of which was recompilation)

# After restarting REPL

julia> using SnoopCompileCore

julia> invalidations = @snoopr using ADRIA end
32262-element Vector{Any}:
  MethodInstance for last(::Base.Slice)
 1
  MethodInstance for last(::Base.Slice)
 2
  MethodInstance for last(::AbstractUnitRange)
  "jl_method_table_insert"
  last(::StaticArrays.SOneTo{n}) where n in StaticArrays at C:\Users\takuy\.julia\packages\StaticArrays\B0HhH\src\SOneTo.jl:39
  "jl_method_table_insert"
 ⋮
  "insert_backedges"
  MethodInstance for ADRIA.run_model(::Domain{NamedArrays.NamedMatrix{Float64, SparseArrays.SparseMatrixCSC{Float64, Int64}, Tuple{OrderedCollections.OrderedDict{String, Int64}, OrderedCollections.OrderedDict{String, Int64}}}, NamedArrays.NamedMatrix{Float32, Matrix{Float32}, Tuple{OrderedCollections.OrderedDict{String, Int64}, OrderedCollections.OrderedDict{String, Int64}}}, Vector{Int64}, DataFrames.DataFrame, String, Vector{Float64}, Vector{String}, NamedArrays.NamedArray{Float32, 3, Array{Float32, 3}, Tuple{OrderedCollections.OrderedDict{String, Int64}, OrderedCollections.OrderedDict{String, Int64}, OrderedCollections.OrderedDict{String, Int64}}}, NamedArrays.NamedArray{Float32, 3, Array{Float32, 3}, Tuple{OrderedCollections.OrderedDict{String, Int64}, OrderedCollections.OrderedDict{String, Int64}, OrderedCollections.OrderedDict{String, Int64}}}, Matrix{Float64}}, ::DataFrames.DataFrameRow{DataFrames.DataFrame, DataFrames.Index}, ::DataFrames.DataFrame, ::NamedTuple{(:sf, :fec_all, :fec_scope, :prop_loss, :Sbl, :dhw_step, :cov_tmp, :felt_dhw, :depth_coeff, :site_area, :TP_data, :waves), Tuple{Matrix{Float64}, Matrix{Float64}, Matrix{Float64}, Matrix{Float64}, Matrix{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, Array{Float64, 3}}})
  "insert_backedges"
  MethodInstance for ADRIA.run_model(::Domain, ::DataFrames.DataFrameRow{DataFrames.DataFrame, DataFrames.Index}, ::DataFrames.DataFrame, ::NamedTuple{(:sf, :fec_all, :fec_scope, :prop_loss, :Sbl, :dhw_step, :cov_tmp, :felt_dhw, :depth_coeff, :site_area, :TP_data, :waves), _A} where _A<:Tuple{Matrix{Float64}, Matrix{Float64}, Matrix{Float64}, Matrix{Float64}, Matrix{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, Vector{Float64}, Any, Matrix{Float64}, Any})
  "insert_backedges"
  MethodInstance for CSV.File(::CSV.Context, ::Bool)
  "insert_backedges"

julia> using SnoopCompile

julia> length(uinvalidated(invalidations))
8816

julia> trees = invalidation_trees(invalidations)
ERROR: ArgumentError: Collection has multiple elements, must contain exactly 1 element
Stacktrace:
 [1] only
   @ .\iterators.jl:1411 [inlined]
 [2] invalidation_trees(list::Vector{Any}; exclude_corecompiler::Bool)
   @ SnoopCompile C:\Users\takuy\.julia\packages\SnoopCompile\Q8zUg\src\invalidations.jl:367
 [3] invalidation_trees(list::Vector{Any})
   @ SnoopCompile C:\Users\takuy\.julia\packages\SnoopCompile\Q8zUg\src\invalidations.jl:294
 [4] top-level scope
   @ REPL[12]:1

[timholy]

Without having access to ADRIA this will be hard for me to debug. Which version of SnoopCompile are you using? For me invalidations.jl: 367 does not currently contain a call to only. I am guessing it's this line:

SnoopCompile.jl/src/invalidations.jl

Line 386 in fa775b3

root = only(root.children)

But you might consider updating to a newer SnoopCompile. Another alternative is to download the beta of Julia 1.9 and try that.

[ConnectedSystems]

Thanks, I'll have a look tomorrow but I was certain all packages are their most recent. Will update after I confirm. I'll also try it with v1.9-beta

[timholy]

They can't possibly be at their most recent (now), given that stacktrace. But three weeks ago, that may well have been true!

[ConnectedSystems]

On Julia v1.8.5, but now with SnoopCompile v2.9.8, the error goes away and the expected output seems to be displayed, albeit with a number of warnings that repeat (27 times to be exact):

julia> trees = invalidation_trees(invalidations)
┌ Warning: MethodInstance for MacroTools.match(::Expr, ::Expr, ::Dict{Any, Any}) not found in `backedge_table`
└ @ SnoopCompile C:\Users\takuy\.julia\packages\SnoopCompile\yStp6\src\invalidations.jl:429
┌ Warning: MethodInstance for MacroTools.match(::Expr, ::Expr) not found in `backedge_table`
└ @ SnoopCompile C:\Users\takuy\.julia\packages\SnoopCompile\yStp6\src\invalidations.jl:429
# ... this repeats 27 times ...

I've been able to move forward now so thank you for fixing whatever was causing my issue. If the warnings are not of concern, I think this issue can be closed.

Thanks again!

[timholy]

You might try Julia 1.9.0-beta (or wait until 1.9 comes out officially), I think that gets rid of the warnings.