Problems with 'loading' types in modules

[wg030]

Hi!
I have some problems with loading saved Interpolants again. My original problem (see #99) dissapeared with a update a few days later.
However right now I am trying to organize my code in modules. My new code as a minimal working example looks as follows.

runfirst.jl:

using JLD2, FileIO
using Interpolations

x = [-1, 0, 1, 2]
y = x.^2
intp = interpolate( (x,), y, Gridded( Linear()))
@save "f.jld2" intp

tmp.jl:

push!( LOAD_PATH, pwd())

using JLD2, FileIO
using Interpolations
using TMPModule

println( "Test: $(intp(1.5))")

TMPModule.jl:

module TMPModule

using JLD2, FileIO
using Interpolations

export intp

#=
# this works as expected!
x = [-1, 0, 1, 2]
y = x.^2
intp = interpolate( (x,), y, Gridded( Linear()))
# =#

# this does not work
@load "f.jld2" intp

end

The @load command in TMPModule causes the following error, whereas if I generated the variable intp as demonstraded in the code that is commented out everything would work as I would love it to.

┌ Warning: type Interpolations.Interpolations.GriddedInterpolation{Float64,1,Int64,Interpolations.Interpolations.Gridded{Interpolations.Interpolations.Linear},Tuple{Array{Int64,1}}} does not exist in workspace; reconstructing
└ @ JLD2 C:\Users\homer.julia\packages\JLD2\KjBIK\src\data.jl:1154

WARNING: eval into closed module ReconstructedTypes:
Expr(:struct_type, Symbol("##Interpolations.Interpolations.GriddedInterpolation{Float64,1,Int64,Interpolations.Interpolations.Gridded{Interpolations.Interpolations.Linear},Tuple{Array{Int64,1}}}#363"), svec(), svec(:knots, :coefs), Any, svec(Tuple{Array{Int64, 1}}, Any), false, 0)
** incremental compilation may be broken for this module **

[photicus]

To me, this looks like the same issue as #107, #122 and #123. What happens if you import Interpolations before including TMPModule.jl ? It seems that JLD2 looks for modules in the current workspace as opposed to the package.

[wg030]

@photicus
Thanks for your answer! Importing Interpolations in the REPL before running tmp.jl did not solve the problem. However with the latest discussion in #107 between you and @qua4tre I could make my code work. The TMPModule now looks as follows:

TMPModule.jl:

module TMPModule

using JLD2, FileIO
#using Interpolations
Core.eval(Main, :(using Interpolations))

export intp

# this works now, but simpliy "using Interpolations" does not
@load "f.jld2" intp

end

I will leave this issue open nevertheless because I suppose this solution is more a workaround than the intended way of doing it.

[lstagner]

I am running into a similar problem. I am trying to use JLD2 to create checkpoints for an optimization function. Using eval isn't working. Here is a MWE

module MyModule

using JLD2
using Sobol
# Core.eval(Main, :(using Sobol)) # This doesn't work

function myfunction(warmstart::Bool)
    s = SobolSeq(2)
    if warmstart
        @load "checkpoint.jld2" s
    end
    next!(s)
    @save "checkpoint.jld2" s
    return s
end

export myfunction

end

julia> push!( LOAD_PATH, pwd());

julia> using MyModule
[ Info: Precompiling MyModule [top-level]

julia> myfunction(false)
2-dimensional Sobol sequence on [0,1]^2

julia> myfunction(true)
┌ Warning: type Sobol.SobolSeq{2} does not exist in workspace; reconstructing
└ @ JLD2 ~/.julia/packages/JLD2/KjBIK/src/data.jl:1153
ERROR: MethodError: no method matching next!(::getfield(JLD2.ReconstructedTypes, Symbol("##Sobol.SobolSeq{2}#363")))
Closest candidates are:
  next!(::Sobol.SobolSeq) at /Users/lstagner/.julia/packages/Sobol/uatKH/src/Sobol.jl:82
  next!(::Sobol.SobolSeq, ::AbstractArray{#s16,1} where #s16<:AbstractFloat) at /Users/lstagner/.julia/packages/Sobol/uatKH/src/Sobol.jl:58
  next!(::Sobol.SobolSeq, ::AbstractArray{#s16,1} where #s16<:AbstractFloat, ::AbstractArray{T,1} where T, ::AbstractArray{T,1} where T) at /Users/lstagner/.julia/packages/Sobol/uatKH/src/Sobol.jl:136
  ...
Stacktrace:
 [1] myfunction(::Bool) at /Users/lstagner/MyModule.jl:11
 [2] top-level scope at none:0

[timholy]

This may be a consequence of Julia no longer importing all modules into Main. If someone wants to fix this, JLD2 might need to inspect Base.module_keys and extract the correct module that way. E.g.,

julia> filter(m->nameof(m) == :Distributed, keys(Base.module_keys))
Set(Module[Distributed])

gets you the "real" Distributed module.

[lstagner]

Thanks for the tip. I have opened a PR with a fix.

[timholy]

👍 In the long run, it might be wise to store the Pkg UUID in the file too.

[AshtonSBradley]

Not sure if this is fixed, but on JLD2#master I am still having this issue.

Mine is exactly #134 (comment)

and solved as discussed there.