JLD failure

[montyvesselinov]

A file generated (JLD.save) with the latest version of julia (1.1) and JLD (v0.9.1) cannot be loaded back using JLD.load. How is this possible?

ERROR: unrecognized HDF5 datatype class 4
Stacktrace:
[1] error(::String, ::Int32) at ./error.jl:42
[2] jldatatype(::JLD.JldFile, ::HDF5.HDF5Datatype) at /Users/monty/.julia/packages/JLD/1BoSz/src/jld_types.jl:738
[3] read(::JLD.JldDataset) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:370
[4] read_ref(::JLD.JldFile, ::HDF5.HDF5ReferenceObj) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:502
[5] read_refs(::JLD.JldDataset, ::Type{Any}, ::Int32, ::Int32, ::Tuple{Int64}) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:475
[6] read_array(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Int32, ::Int32, ::Tuple{Int64}) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:409
[7] read_array(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Int32, ::Int32) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:408
[8] read(::JLD.JldDataset) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:372
[9] read_ref(::JLD.JldFile, ::HDF5.HDF5ReferenceObj) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:502
[10] jlconvert(::Type{JLD.AssociativeWrapper{Symbol,Any,Dict{Symbol,Any}}}, ::JLD.JldFile, ::Ptr{UInt8}) at /Users/monty/.julia/packages/JLD/1BoSz/src/jld_types.jl:434
[11] read_scalar(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Type) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:398
[12] read(::JLD.JldDataset) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:370
[13] read_ref(::JLD.JldFile, ::HDF5.HDF5ReferenceObj) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:502
[14] jlconvert(::Type{getfield(JLD, Symbol("##TensorDecompositions.Tucker{Core.Float64,3}#21182"))}, ::JLD.JldFile, ::Ptr{UInt8}) at /Users/monty/.julia/packages/JLD/1BoSz/src/jld_types.jl:434
[15] read_scalar(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Type) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:398
[16] read(::JLD.JldDataset) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:370
[17] read_ref(::JLD.JldFile, ::HDF5.HDF5ReferenceObj) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:502
[18] read_refs(::JLD.JldDataset, ::Type{Any}, ::Int32, ::Int32, ::Tuple{Int64}) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:475
[19] read_array(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Int32, ::Int32, ::Tuple{Int64}) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:409
[20] read_array(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Int32, ::Int32) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:408
[21] read(::JLD.JldDataset) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:372
[22] read(::JLD.JldFile, ::String) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:346
[23] #42 at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:1240 [inlined]
[24] #jldopen#14(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(JLD, Symbol("##42#43")){String}, ::String, ::Vararg{String,N} where N) at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:246
[25] jldopen at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:244 [inlined]
[26] load at /Users/monty/.julia/packages/JLD/1BoSz/src/JLD.jl:1239 [inlined]
[27] #load#13(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /Users/monty/.julia/packages/FileIO/YJO7Z/src/loadsave.jl:118
[28] load at /Users/monty/.julia/packages/FileIO/YJO7Z/src/loadsave.jl:118 [inlined]

[loleg]

+1 - same issue when saving an array loaded from JSON (containing Dicts, strings, ints), working inside of IJulia with Julia 1.0.3

[githubtomtom]

+1
same issue: "ERROR: unrecognized HDF5 datatype class 4" loading an object of my defined type.

[rasmushenningsson]

I get the same error in both windows 10 and ubuntu (using virtualbox) when running the Unit Tests for JLD. It fails on Bools (see #251).

The good news is that the bug does not appear for me on master. Could someone make a new release?

[hronellenfitsch]

I am having the same issue, with a jld file containing Dicts.
Julia 1.1.1 and latest JLD 0.9.1.

[Azenor]

I get the same error, I ran some simulations on a cluster and saved them in a jld file, containing Dicts as well. When trying to open the file either back on my computer (ubuntu, julia v1.1.0) or directly on the cluster I get this issue. Any solutions so far? Does it do the same with JLD2?

[hronellenfitsch]

@Azenor I couldn't get JLD to work, but had no trouble with BSON.jl. That might be a good alternative until JLD is fixed.

[Azenor]

thank you @hronellenfitsch ! I tried with BSON but my object is an array of Dicts actually so it did not work, I used JLD2 in stead it seems to work (just got this warning due to the type of one of my variables : "Warning: type LinearAlgebra.Adjoint{Float64,Array{Float64,2}} does not exist in workspace; reconstructing").

[iakovost]

Do we have an update on this issue?

[Sinansi]

This error is headache. I have lost 21 hours of work due to this error.
So what is causing it? Is it due to saving a variable of boolean type?

[timholy]

Someone needs to supply a reproducible test case, otherwise it's essentially unfixable.

[Sinansi]

Someone on stackoverflow says use JLD2 instead of JLD. He said, saving the file with extension .jld2 instead of jld solved the issue. Is JLD2 supposed to be an improved replacement of JLD?
https://github.com/JuliaIO/JLD2.jl

[timholy]

Yes, though neither is perfect. Best practice is to report bugs (with a full reproducer) whenever you encounter a problem, so that over time whatever package you use will get better.

[rasmushenningsson]

Someone needs to supply a reproducible test case, otherwise it's essentially unfixable.

It is reproducible in JLD v0.9.1 - the unit tests fail. (I've seen it on multiple computers, windows 10 and ubuntu in virtualbox.)

It has been fixed on JLD master (#269), but no new release has been made.

[ericrouviere]

My problem was fixed by updating JLD on Julia 1.3.1

julia> Pkg.update("JLD")
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed JLD ─ v0.9.2
  Updating `~/.julia/environments/v1.3/Project.toml`
  [4138dd39] ↑ JLD v0.9.1 ⇒ v0.9.2
  Updating `~/.julia/environments/v1.3/Manifest.toml`
  [4138dd39] ↑ JLD v0.9.1 ⇒ v0.9.2
  [9abbd945] - Profile 

Before the update saving Any arrays with bool elements caused this problem. Similar to #269 .

julia> M = Any[false, rand(1)]
2-element Array{Any,1}:
 false                    
      [0.7697653460076719]

julia> save("M.jld", "M", M)

julia> load("M.jld")
ERROR: unrecognized HDF5 datatype class 4
Stacktrace:
 [1] error(::String, ::Int32) at ./error.jl:42
 [2] jldatatype(::JLD.JldFile, ::HDF5.HDF5Datatype) at /home/erouviere/.julia/packages/JLD/1BoSz/src/jld_types.jl:738
 [3] read(::JLD.JldDataset) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:370
 [4] read_ref(::JLD.JldFile, ::HDF5.HDF5ReferenceObj) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:502
 [5] read_refs(::JLD.JldDataset, ::Type{Any}, ::Int64, ::Int64, ::Tuple{Int64}) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:475
 [6] read_array(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Int64, ::Int64, ::Tuple{Int64}) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:409
 [7] read_array(::JLD.JldDataset, ::HDF5.HDF5Datatype, ::Int64, ::Int64) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:408
 [8] read(::JLD.JldDataset) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:372
 [9] read(::JLD.JldFile, ::String) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:346
 [10] #39 at ./none:0 [inlined]
 [11] iterate at ./generator.jl:47 [inlined]
 [12] collect(::Base.Generator{Array{String,1},JLD.var"#39#41"{JLD.JldFile}}) at ./array.jl:622
 [13] (::JLD.var"#38#40")(::JLD.JldFile) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:1234
 [14] #jldopen#14(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(jldopen), ::JLD.var"#38#40", ::String, ::Vararg{String,N} where N) at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:246
 [15] load at /home/erouviere/.julia/packages/JLD/1BoSz/src/JLD.jl:244 [inlined]
 [16] #load#13(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(load), ::String) at /home/erouviere/.julia/packages/FileIO/I1ONY/src/loadsave.jl:118
 [17] load(::String) at /home/erouviere/.julia/packages/FileIO/I1ONY/src/loadsave.jl:118
 [18] top-level scope at REPL[8]:1