Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Julia 1.7: Cannot load StepRangeLen object saved with Julia 1.6 #359

Closed
getzze opened this issue Dec 6, 2021 · 4 comments
Closed

Julia 1.7: Cannot load StepRangeLen object saved with Julia 1.6 #359

getzze opened this issue Dec 6, 2021 · 4 comments
Labels

Comments

@getzze
Copy link

getzze commented Dec 6, 2021

Julia: v1.7.0
JLD2.jl: v0.4.15

I have some data that I saved as a "data.jl2" file with Julia 1.6 (I probably saved it with julia 1.3 actually but it shouldn't matter) and I cannot read it with the new julia version 1.7. I get the following error:

┌ Warning: read type StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}} is not a leaf type in workspace; reconstructing
└ @ JLD2 ~/.julia/packages/JLD2/DcnTD/src/data/reconstructing_datatypes.jl:231
Error encountered while load FileIO.File{FileIO.DataFormat{:JLD2}, String}("data.jld2").

Fatal error:

TypeError: in <:, expected Type, got a value of type JLD2.OnDiskRepresentation{(0, 16, 32, 40), Tuple{Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64, Int64}, Tuple{Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64, Int64}}

The data is actually a DataFrame, where one column has element types from a custom struct named DataSetGcamp.
This struct is holding the data is various attributes, being dictionaries or dataframes.

It seems the problem comes from loading StepRangeLen types (most probably LinRange is affected also) because the struct parameters changed in julia1.7: JuliaLang/julia#41619
It now has 4 parameters and just 3 in julia 1.6.

I tried Custom Serialization to no avail:

StepRangeLenInt{T,R,S} = StepRangeLen{T,R,S,Int} where {T,R,S}
JLD2.wconvert(::Type{StepRangeLenInt}, a::StepRangeLen) = StepRangeLenInt(a.ref, a.step, a.len, a.offset)
JLD2.rconvert(::Type{StepRangeLen}, a::StepRangeLenInt) = StepRangeLen(a.ref, a.step, a.len, a.offset)

Any other idea would be much appreciated!

Full traceback:
https://gist.github.com/getzze/fe069212fb4bb2a510422a76bce33630

@JonasIsensee
Copy link
Collaborator

Hi @getzze,

I'm not sure, there is a good solution for you.
I'd say, it's probably easiest to continue using julia v1.6 for this project if that is an option.
Alternatively, I'd recommend you load the data in julia v1.6 and convert it to not use those datatypes and re-save.

Since this is a change in a Base-type, many of the typical workarounds probably won't work.

Nonetheless, the error message by itself is odd and I'll classify this as as bug.

@timholy
Copy link
Member

timholy commented Dec 8, 2021

@JonasIsensee
Copy link
Collaborator

Interesting, that looks very similar to the type remapping that I implemented for #316.
There, I needed it since anonymous functions really shouldn't be reconstructed with their original name.
Though, I did that on a by-file basis.

I suppose, one could add that feature independently, even if #316 or #338 will probably never be finished.

@JonasIsensee
Copy link
Collaborator

#376
adds explicit type remapping (reconstructing to a differently named but content-identical datatype)
This is probably the best solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants