Error requiring `HDF5` from `Plots`: Group not defined #3235

StevenWhitaker · 2021-01-23T14:41:56Z

I get the following warning with HDF5.jl v0.13.6 and Plots.jl v1.10.1:

julia> using HDF5

julia> using Plots
WARNING: could not import HDF5.Group into _hdf5_implementation
WARNING: could not import HDF5.Dataset into _hdf5_implementation
┌ Warning: Error requiring `HDF5` from `Plots`
│   exception =
│    LoadError: UndefVarError: Group not defined
│    Stacktrace:
│     [1] top-level scope at /home/steven/.julia/packages/Plots/lmp2A/src/backends/hdf5.jl:162
│     [2] include(::Function, ::Module, ::String) at ./Base.jl:380
│     [3] include at ./Base.jl:368 [inlined]
│     [4] include(::String) at /home/steven/.julia/packages/Plots/lmp2A/src/Plots.jl:1
│     [5] top-level scope at /home/steven/.julia/packages/Plots/lmp2A/src/init.jl:35
│     [6] eval at ./boot.jl:331 [inlined]
│     [7] eval at /home/steven/.julia/packages/Plots/lmp2A/src/Plots.jl:1 [inlined]
│     [8] (::Plots.var"#251#284")() at /home/steven/.julia/packages/Requires/035xH/src/require.jl:94
│     [9] err(::Any, ::Module, ::String) at /home/steven/.julia/packages/Requires/035xH/src/require.jl:42
│     [10] (::Plots.var"#250#283")() at /home/steven/.julia/packages/Requires/035xH/src/require.jl:93
│     [11] withpath(::Any, ::String) at /home/steven/.julia/packages/Requires/035xH/src/require.jl:32
│     [12] (::Plots.var"#249#282")() at /home/steven/.julia/packages/Requires/035xH/src/require.jl:92
│     [13] listenpkg(::Any, ::Base.PkgId) at /home/steven/.julia/packages/Requires/035xH/src/require.jl:15
│     [14] macro expansion at /home/steven/.julia/packages/Requires/035xH/src/require.jl:90 [inlined]
│     [15] __init__() at /home/steven/.julia/packages/Plots/lmp2A/src/init.jl:33
│     [16] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:697
│     [17] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:782
│     [18] _require(::Base.PkgId) at ./loading.jl:1007
│     [19] require(::Base.PkgId) at ./loading.jl:928
│     [20] require(::Module, ::Symbol) at ./loading.jl:923
│     [21] eval(::Module, ::Any) at ./boot.jl:331
│     [22] eval_user_input(::Any, ::REPL.REPLBackend) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:134
│     [23] repl_backend_loop(::REPL.REPLBackend) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:195
│     [24] start_repl_backend(::REPL.REPLBackend, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:180
│     [25] run_repl(::REPL.AbstractREPL, ::Any; backend_on_current_task::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:292
│     [26] run_repl(::REPL.AbstractREPL, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
│     [27] (::Base.var"#807#809"{Bool,Bool,Bool,Bool})(::Module) at ./client.jl:399
│     [28] #invokelatest#1 at ./essentials.jl:710 [inlined]
│     [29] invokelatest at ./essentials.jl:709 [inlined]
│     [30] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:383
│     [31] exec_options(::Base.JLOptions) at ./client.jl:313
│     [32] _start() at ./client.jl:506
│    in expression starting at /home/steven/.julia/packages/Plots/lmp2A/src/backends/hdf5.jl:162
└ @ Requires ~/.julia/packages/Requires/035xH/src/require.jl:44

It looks like backends/hdf5.jl assumes a certain version of HDF5.jl, one in which Group and Dataset structs exist. The version of HDF5.jl I am using (v0.13.6), though, still has HDF5Group and HDF5Dataset structs (it looks like some later version renamed them, not sure which version). Is there some way to make backends/hdf5.jl still work with the earlier versions of HDF5.jl? If not, is there some way to make the warning more brief and informative (e.g., "the HDF5 backend requires HDF5.jl v0.15 or greater" or something like that)? Or is there a way to make the warning appear only when I try to actually use the HDF5 backend?

I guess my main reason for raising this issue is because there's a package I use that depends on both Plots.jl and HDF5.jl, and every time I load the package I get this obtrusive warning that I actually don't care about because I'm not going to use the HDF5 backend anyway.

Here's my versioninfo():

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 8

The text was updated successfully, but these errors were encountered:

mkborregaard · 2021-01-24T11:32:15Z

@ma-laforge would you have the time to have a quick look at this?

ma-laforge · 2021-01-24T18:37:53Z

Hmmm. That's new. I'll have to look into it.

ma-laforge · 2021-01-24T22:12:45Z

Status

The HDF5.jl API changed recently - probably around v0.15.0 (jump in minor revision).
HDF5.jl apparently went though a compat deprecation cycle.
The community updated Plots.jl to reflect these changes (Thank you):
-->Issue: Deprecation warnings for HDF5 backend #3157, PR: Handling Deprecation warnings for HDF5 backend #3175

Cause

People (ex: @StevenWhitaker) want to use the newest Plots.jl with an old version of HDF5.jl

Why is this not a problem with other backends

People typically only use plotting backend modules through Plots.jl itself (they won't use other backends in other parts of their code). So people naturally accept updating to the newest version of the backend package when Plots.jl tracks changes to their interfaces.

However it is reasonable that many people don't intend on using the HDF5 backend - whilst still using HDF5.jl for other purposes in their code.

Options

Here are the most practical options, as far as I can see.

Option 1 (Personal suggestion)

Accept that users might get confused with this message/transition (doesn't happen very often), and suggest they either migrate to the new HDF5.jl, or stick with earlier versions of Plots.jl.

Option 2

Add compat statements here in Plots.jl to maintain support for the deprecated API (older versions of HDF5.jl). I'm not a big fan of this. HDF5.jl already went through its deprecation cycle - I don't think we should prolong it.

Option 3

Add HDF5.jl as an explicit dependency of Plots.jl so that we quickly/easily get more useful error messages (Not a big fan of adding unnecessary dependencies).

Option 4

Perform version check @ startup to ensure we can give a more useful message to the user during this time of transition.

NOTE

This will slow down time to first plot a bit while we scan through Julia's dependency info.
Add complexity to Plots.jl code which I am not convinced is warranted for this problem.
I've done this before for Plots.jl <--> InspectDR compatibility, and it doesn't work particularly well (esp. not in identifying future compatibility issues).
There might be an elegant way to add this feature, but my own solution felt sort of flaky.

Conclusion

As stated above, I vote for option 1 - but it all depends on the philosophy of the Plots.jl team.

Question for the team

What do you suggest we/I do?

mkborregaard · 2021-01-25T09:14:53Z

I vote for option 1 too.

daschw · 2021-01-25T09:27:57Z

I agree, thanks a lot @ma-laforge !

alicecorbella · 2021-05-05T11:57:13Z

Hi @ma-laforge, thank you for this! Apologies if this question is trivial, I am a new user.
What do you mean by "migrate to the new HDF5.jl"? I have the most updated version of both packages but I seems to get this error still. Could you clarify that please?
Thanks in advance!

ma-laforge · 2021-05-05T14:15:55Z

What's the "newest" version?

@alicecorbella: If you go to HDF5.jl's website (https://github.com/JuliaIO/HDF5.jl), you'll see the version number of the newest release in the "Releases" section. At the time of writing, that's v0.15.4.

To be fair... that's assuming the maintainers are diligent about creating Git "Releases" or have set up their package repository for auto-generating them. Thankfully, this is appears to typically be the case.

Do I have the newest version?

You might think you have the newest version, but certain packages require older versions of packages you've already installed... So adding them to your Julia installation[1] might actually end up downgrading some of the packages you've already installed.

[1] The more accurate statement would be: "to your current Julia environment".

To see what version of HDF5 you are actually using, you can run "status" in Julia's package manager:

julia> ]
pkg> st

If you can't see HDF5, you can either:

pkg> add HDF5 which will make HDF5 directly available to you (the user) - and allow you see the installed version the next time you show the package environment's "status".
Look at the Manifest.toml file of your current environment to see what version of HDF5 is being used
I know of other ways, but it is mostly useful if you need to check the version number from within your code.

Hope that helps

mattcbro · 2021-12-10T05:33:26Z

This is giving me an error which breaks my code after upgrading to Julia 1.7. I see the following:

Starting Julia...
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0 (2021-11-30)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using Revise

julia> using HDF5

julia> using Plots
WARNING: could not import Plots.HDF5 into _hdf5_implementation
ERROR: InitError: UndefVarError: HDF5 not defined
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/Plots/FCM0H/src/backends/hdf5.jl:48
Revise evaluation error at /home/matt/.julia/packages/Plots/FCM0H/src/backends/hdf5.jl:48

Stacktrace:
  [1] methods_by_execution!(recurse::Any, methodinfo::Revise.CodeTrackingMethodInfo, docexprs::Dict{Module, Vector{Expr}}, mod::Module, ex::Expr; mode::Symbol, disablebp::Bool, always_rethrow::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Revise ~/.julia/packages/Revise/WHZdV/src/lowered.jl:232
  [2] #eval_with_signatures#92
    @ ~/.julia/packages/Revise/WHZdV/src/packagedef.jl:470 [inlined]
  [3] instantiate_sigs!(modexsigs::OrderedCollections.OrderedDict{Module, OrderedCollections.OrderedDict{Revise.RelocatableExpr, Union{Nothing, Vector{Any}}}}; mode::Symbol, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Revise ~/.julia/packages/Revise/WHZdV/src/packagedef.jl:478
  [4] #invokelatest#2
    @ ./essentials.jl:718 [inlined]
  [5] maybe_add_includes_to_pkgdata!(pkgdata::Revise.PkgData, file::String, includes::Vector{Pair{Module, String}}; eval_now::Bool)
    @ Revise ~/.julia/packages/Revise/WHZdV/src/pkgs.jl:174
  [6] eval_require_now(pkgdata::Revise.PkgData, fileidx::Int64, filekey::String, sourcefile::String, modcaller::Module, expr::Expr)
    @ Revise ~/.julia/packages/Revise/WHZdV/src/pkgs.jl:288
  [7] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
  [8] invokelatest
    @ ./essentials.jl:714 [inlined]
  [9] add_require(sourcefile::String, modcaller::Module, idmod::String, modname::String, expr::Expr)
    @ Revise ~/.julia/packages/Revise/WHZdV/src/pkgs.jl:233
 [10] withnotifications(::Any, ::Vararg{Any})
    @ Requires ~/.julia/packages/Requires/7Ncym/src/require.jl:68
 [11] (::Plots.var"#296#332")()
    @ Plots ~/.julia/packages/Requires/7Ncym/src/require.jl:96
 [12] listenpkg(f::Any, pkg::Base.PkgId)
    @ Requires ~/.julia/packages/Requires/7Ncym/src/require.jl:20
 [13] macro expansion
    @ ~/.julia/packages/Requires/7Ncym/src/require.jl:95 [inlined]
 [14] __init__()
    @ Plots ~/.julia/packages/Plots/FCM0H/src/init.jl:44
 [15] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:768
 [16] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:854
 [17] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1097
 [18] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
 [19] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
during initialization of module Plots

The one added difference is that I did a using call for the Revise package first. Also I believe I have the "latest" HDF5 package, namely

(@v1.7) pkg> st
      Status `~/.julia/environments/v1.7/Project.toml`
  [28f2ccd6] ApproxFun v0.13.0
  [7d9fca2a] Arpack v0.4.0
  [c52e3926] Atom v0.12.35
  [6e4b80f9] BenchmarkTools v1.2.2
  [336ed68f] CSV v0.9.11
  [159f3aea] Cairo v1.0.5
  [a10d1c49] DBInterface v2.5.0
  [717857b8] DSP v0.7.4
  [a93c6f00] DataFrames v1.3.0
  [864edb3b] DataStructures v0.18.11
  [31a5f54b] Debugger v0.6.8
  [7806a523] DecisionTree v0.10.11
  [39dd38d3] Dierckx v0.5.1
  [7a1cc6ca] FFTW v1.4.5
  [5789e2e9] FileIO v1.11.2
  [186bb1d3] Fontconfig v0.4.0
  [9dda63f9] ForceImport v0.0.3
  [c27321d9] Glob v1.3.0
  [f67ccb44] HDF5 v0.15.7
  [5903a43b] Infiltrator v1.0.3
  [d0351b0e] InspectDR v0.4.3
  [a98d9a8b] Interpolations v0.13.4
  [033835bb] JLD2 v0.4.15
  [e5e0dc1b] Juno v0.8.4
  [ba0b0d4f] Krylov v0.7.9
  [23992714] MAT v0.10.1
  [add582a8] MLJ v0.16.11
  [39abe10b] MySQL v1.2.1
  [9b87118b] PackageCompiler v2.0.3
  [d96e819e] Parameters v0.12.3
  [91a5bcdd] Plots v1.25.2
  [f27b6e38] Polynomials v2.0.18
  [438e738f] PyCall v1.92.5
  [d330b81b] PyPlot v2.10.0
  [93e0c654] Reduce v1.2.11 `https://github.com/chakravala/Reduce.jl.git#master`
  [bfcbc1eb] ReduceLinAlg v0.1.0
  [295af30f] Revise v3.2.0
  [3646fa90] ScikitLearn v0.6.4
  [276daf66] SpecialFunctions v1.8.1
  [90137ffa] StaticArrays v1.2.13
  [f3b207a7] StatsPlots v0.14.29
  [9ff05d80] TickTock v1.1.0
  [009559a3] XGBoost v1.1.1
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles

Should I open this up as a new bug?

BeastyBlacksmith · 2021-12-10T08:13:34Z

Since this error is thrown by Revise I'd open the issue there

mattcbro · 2021-12-10T21:18:51Z

Since this error is thrown by Revise I'd open the issue there

I think it might be related to an open issue already reported there for Revise 3.2.0
timholy/Revise.jl#658 (comment)

bashartalafha · 2022-09-16T07:35:25Z

Try to run your code on Colab notebook.

JeffFessler mentioned this issue Jan 24, 2021

package update / new HDF5 / github actions JeffFessler/MIRTio.jl#10

Merged

StevenWhitaker mentioned this issue May 21, 2021

new error related to HDF5 JeffFessler/MIRT.jl#93

Closed

BeastyBlacksmith closed this as completed Jun 7, 2021

haakon-e mentioned this issue Sep 14, 2022

SeisIO prevents other packages to be updated jpjones76/SeisIO.jl#85

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error requiring `HDF5` from `Plots`: Group not defined #3235

Error requiring `HDF5` from `Plots`: Group not defined #3235

StevenWhitaker commented Jan 23, 2021

mkborregaard commented Jan 24, 2021

ma-laforge commented Jan 24, 2021

ma-laforge commented Jan 24, 2021

mkborregaard commented Jan 25, 2021

daschw commented Jan 25, 2021

alicecorbella commented May 5, 2021

ma-laforge commented May 5, 2021

mattcbro commented Dec 10, 2021

BeastyBlacksmith commented Dec 10, 2021

mattcbro commented Dec 10, 2021 •

edited

Loading

bashartalafha commented Sep 16, 2022

Error requiring HDF5 from Plots: Group not defined #3235

Error requiring HDF5 from Plots: Group not defined #3235

Comments

StevenWhitaker commented Jan 23, 2021

mkborregaard commented Jan 24, 2021

ma-laforge commented Jan 24, 2021

ma-laforge commented Jan 24, 2021

Status

Cause

Why is this not a problem with other backends

Options

Option 1 (Personal suggestion)

Option 2

Option 3

Option 4

Conclusion

Question for the team

mkborregaard commented Jan 25, 2021

daschw commented Jan 25, 2021

alicecorbella commented May 5, 2021

ma-laforge commented May 5, 2021

What's the "newest" version?

Do I have the newest version?

mattcbro commented Dec 10, 2021

BeastyBlacksmith commented Dec 10, 2021

mattcbro commented Dec 10, 2021 • edited Loading

bashartalafha commented Sep 16, 2022

Error requiring `HDF5` from `Plots`: Group not defined #3235

Error requiring `HDF5` from `Plots`: Group not defined #3235

mattcbro commented Dec 10, 2021 •

edited

Loading