cannot write a pointer to JLD file error when trying to save DiffEq solution from certain problem types

[TorkelE]

I am looking at biochemical reaction networks. To help with this we have a certain @rection_network macro to help us easily create such models. These then creates an AbstractReactionNetwork type of object, these are a subtype of functions and can be used as input to various problems from DiffEq.

Now the problem is that I am unable to save solutions originating from such a problem, but not those ordinarily created by DIffEq. This works fine:

using OrdinaryDiffEq, JLD2
f(u,p,t) = 1.01*u
u0=1/2
tspan = (0.0,1.0)
prob = ODEProblem(f,u0,tspan)
sol = solve(prob,Tsit5(),reltol=1e-8,abstol=1e-8)
@save "out.jld2" sol

however, this

using DifferentialEquations, JLD2
rn = @reaction_network begin
    (1,10), X ↔ ∅
end
prob = ODEProblem(rn,[20.],(0.,10.))
sol = solve(prob,Rosenbrock23())
@save "out.jld2" sol

yields a big error:

cannot write a pointer to JLD file

Stacktrace:
 [1] h5fieldtype(::JLD2.JLDFile{JLD2.MmapIO}, ::Type{Ptr{Nothing}}, ::Type, ::Type{Val{true}}) at /home/user.name/.julia/packages/JLD2/KjBIK/src/data.jl:994
 [2] commit_compound(::JLD2.JLDFile{JLD2.MmapIO}, ::Array{Symbol,1}, ::DataType, ::Type) at /home/user.name/.julia/packages/JLD2/KjBIK/src/data.jl:185
 [3] h5type(::JLD2.JLDFile{JLD2.MmapIO}, ::Type, ::SymEngine.Basic) at /home/user.name/.julia/packages/JLD2/KjBIK/src/data.jl:163
 [4] h5type at /home/user.name/.julia/packages/JLD2/KjBIK/src/data.jl:168 [inlined]
 [5] write_dataset at /home/user.name/.julia/packages/JLD2/KjBIK/src/datasets.jl:521 [inlined]
 [6] write_ref_mutable at /home/user.name/.julia/packages/JLD2/KjBIK/src/datasets.jl:526 [inlined]
 [7] write_ref at /home/user.name/.julia/packages/JLD2/KjBIK/src/datasets.jl:534 [inlined]
 [8] h5convert! at /home/user.name/.julia/packages/JLD2/KjBIK/src/data.jl:658 [inlined]
 [9] write_data(::JLD2.MmapIO, ::JLD2.JLDFile{JLD2.MmapIO}, ::Array{SymEngine.Basic,2}, ::Type{JLD2.RelOffset}, ::JLD2.HasReferences, ::JLD2.JLDWriteSession{Dict{UInt64,JLD2.RelOffset}}) at /home/user.name/.julia/packages/JLD2/KjBIK/src/dataio.jl:178

...


 [32] (::getfield(Main, Symbol("##36#37")))(::JLD2.JLDFile{JLD2.MmapIO}) at /home/user.name/.julia/packages/JLD2/KjBIK/src/loadsave.jl:50
 [33] #jldopen#31(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(Main, Symbol("##36#37")), ::String, ::Vararg{String,N} where N) at /home/user.name/.julia/packages/JLD2/KjBIK/src/loadsave.jl:4
 [34] jldopen(::Function, ::String, ::String) at /home/user.name/.julia/packages/JLD2/KjBIK/src/loadsave.jl:2
 [35] top-level scope at /home/user.name/.julia/packages/JLD2/KjBIK/src/loadsave.jl:48
 [36] top-level scope at In[1]:6

[ChrisRackauckas]

Looks like this works now:

using OrdinaryDiffEq, JLD2
u0=1/2
tspan = (0.0,1.0)
prob = ODEProblem{false}(f,u0,tspan)
sol = solve(prob,Tsit5(),reltol=1e-8,abstol=1e-8)
@save "out.jld2" sol

# New session
using OrdinaryDiffEq, JLD2
f(u,p,t) = 1.01*u
@load "out.jld2" sol
sol(0.5)

[ChrisRackauckas]

Can probably be closed.

[DilumAluthge]

@ChrisRackauckas Would also be great if you could add a regression test for this one.

[ChrisRackauckas]

The regression test from #122 would cover this. It has the two big features of the DESolution types:

1. The call overloaded function overloads that need to be added from the module
2. The ability to put a generic function into the type and have it unload without breaking, and reload knowing the function.

[dcelisgarza]

This seems to still be an issue with SuiteSparse.CHOLMOD.Factor{Float64}. It may also be an issue with other SuiteSparse factorisations but i haven't tried them. I have a structure that keeps a matrix of this type, but i'll just include saving the matrix because the rest is fluff.

julia> save("out.jld2", "A", A)

ERROR: cannot write a pointer to JLD file
Stacktrace:
 [1] handle_error(::JLD2.PointerException, ::File{DataFormat{:JLD2}}) at C:\Users\Daniel Celis Garza\.julia\packages\FileIO\TyKdX\src\error_handling.jl:82
 [2] handle_exceptions(::Array{Any,1}, ::String) at C:\Users\Daniel Celis Garza\.julia\packages\FileIO\TyKdX\src\error_handling.jl:77
 [3] save(::Formatted, ::Any, ::Vararg{Any,N} where N; options::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\Daniel Celis Garza\.julia\packages\FileIO\TyKdX\src\loadsave.jl:238
 [4] save at C:\Users\Daniel Celis Garza\.julia\packages\FileIO\TyKdX\src\loadsave.jl:217 [inlined]
 [5] #save#19 at C:\Users\Daniel Celis Garza\.julia\packages\FileIO\TyKdX\src\loadsave.jl:139 [inlined]
 [6] save(::String, ::String, ::SuiteSparse.CHOLMOD.Factor{Float64}) at C:\Users\Daniel Celis Garza\.julia\packages\FileIO\TyKdX\src\loadsave.jl:139
 [7] top-level scope at REPL[10]:1

It's really not that big a problem as the matrix can be reconstructed from other data, it just makes it a bit of a pain to save the whole data structure that stores it.

I had a similar issue with JSON.jl but all i had to do was change the lowering function for the offending pointer to nothing and accept that i just have to reconstruct the data later on.

Serialization.serialize also can't keep track of this pointer. It gets deferenced during serialising but still saves the file. Serialization.deserialize can load the file and create the structure where the matrix is stored but spits out a non-blocking error that states the pointer is null. Again only showing what happens if i save the matrix because the rest is working fine.

julia> using Serialize

julia> serialize("A.out", A)

julia> A = deserialize("out.out")

SuiteSparse.CHOLMOD.Factor{Float64}
Error showing value of type SuiteSparse.CHOLMOD.Factor{Float64}:
ERROR: ArgumentError: pointer to the SuiteSparse.CHOLMOD.C_Factor{Float64} object is null. This can happen if the object has been serialized.
Stacktrace:
 [1] unsafe_convert at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\SuiteSparse\src\cholmod.jl:371 [inlined]
 [2] pointer at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\SuiteSparse\src\cholmod.jl:379 [inlined]
 [3] showfactor(::IOContext{REPL.Terminals.TTYTerminal}, ::SuiteSparse.CHOLMOD.Factor{Float64}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\SuiteSparse\src\cholmod.jl:1135
 [4] show at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\SuiteSparse\src\cholmod.jl:1126 [inlined]
 [5] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::SuiteSparse.CHOLMOD.Factor{Float64}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\SuiteSparse\src\cholmod.jl:1147        
 [6] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:214
 [7] display(::REPL.REPLDisplay, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:218
 [8] display(::Any) at .\multimedia.jl:328
 [9] #invokelatest#1 at .\essentials.jl:710 [inlined]
 [10] invokelatest at .\essentials.jl:709 [inlined]
 [11] print_response(::IO, ::Any, ::Bool, ::Bool, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:238
 [12] print_response(::REPL.AbstractREPL, ::Any, ::Bool, ::Bool) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:223
 [13] (::REPL.var"#do_respond#54"{Bool,Bool,VSCodeServer.var"#40#41"{REPL.LineEditREPL,REPL.LineEdit.Prompt},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:822
 [14] #invokelatest#1 at .\essentials.jl:710 [inlined]
 [15] invokelatest at .\essentials.jl:709 [inlined]
 [16] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\LineEdit.jl:2355
 [17] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:1144
 [18] (::REPL.var"#38#42"{REPL.LineEditREPL,REPL.REPLBackendRef})() at .\task.jl:356

Perhaps this is how JLD2 should treat objects such as these.

To reiterate, not that big an issue, i'd just would like the convenience of saving the whole structure even if saving the matrix fails. Also worth checking whether this happens with other SuiteSparse matrices. Taking a similar approach to the built-in serialiser might be the way to go.