HDF5-DIAG warnings from libnetcdf v4.9.1

[trexfeathers]

📰 Custom Issue

Since #5177, NetCDF loading is prompting verbose HDF5-DIAG warnings in some/all cases; here is an example.

@pp-mo and I have done some testing and isolated this to the advancement of libnetcdf from v4.8.1 to v4.9.1.

As far as we can tell this is probably innocuous, a view tentatively shared in pydata/xarray#7549. However it is going to start alarming developers and users alike, and will be a source of frustration given how verbose Iris' own warnings are as well. We've already had a case where it gets in the way of other print statements and thus makes ordinary work that much harder.

So far not sure what the next step is, but wanted to post this to forewarn everyone else.

[pp-mo]

advancement of libnetcdf from v4.8.1 to v4.9.1

Again tentatively, from pydata/xarray#7549 it looks like the actual problem may lie in hdf5 changes introduced with hdf5 v1.12.2
That version of hdf5 has already been out for months, but we suspect the libnetcdf changes may have newly "exposed" the problem.

[pp-mo]

I had a lot of these errors in #5191, as you might expect.
See : these test runs, raw log here

To be fair, I don't think that caused any of the test failures there, since the few actual failures remained the same when I pinned libnetcdf
-- and pinning to libnetcdf=4.8 removed all the offending HDF5 warnings, as expected

[pp-mo]

I don't think that caused any of the test failures there ... and pinning to libnetcdf=4.8 removed all the offending HDF5 warnings

However, I do now seem to have python 3.10 specific failures.
Which I think may indicate that pinning libnetcdf is not a panacea as it may introduce actual errors 😬

Update 2023-03-14: it is in fact not confined to Python 3.10. It seems now to work, with a fix, akin to that in #5095

[pp-mo]

Ping !

Latest main-branch doctests are really full of the HDF5-DIAG warning messages
E.G. : https://pipelines.actions.githubusercontent.com/serviceHosts/85610251-844a-4a1d-8171-bd20e30f9c14/_apis/pipelines/1/runs/4455/signedlogcontent/6?urlExpires=2023-03-27T00%3A35%3A51.0247240Z&urlSigningMethod=HMACV1&urlSignature=9tcPZXK%2B%2Bov5smdX5IBtQ4z5s41u5Y%2BANwxBJcsfJOc%3D

Could this also be the basic reason we are getting benchmark regressions ?
We really need to seek a solution to this.

[trexfeathers]

Could this also be the basic reason we are getting benchmark regressions ?

It would explain the regressions in #5182, but not the others, since libnetcdf has not advanced since then.

We really need to seek a solution to this.

Since the core devs are assigned to other work for the next two weeks, we would need to explicitly drop something to make room for this investigation. We can discuss that this morning if you think it is sensible.

[trexfeathers]

Poll:

• Iris 3.5 libnetcdf pin? #5213

[trexfeathers]

Relevant: pydata/xarray#7388 (comment)

[zklaus]

I resolved this in the conda-forge package for libnetcdf 4.9.2 in conda-forge/libnetcdf-feedstock#175. Would it be useful to have it for 4.9.1 as well? The backport would be easy enough, but it's always a little bit of a hassle to release older versions. Would pinning to !=4.9.1 be an option?

[trexfeathers]

I resolved this in the conda-forge package for libnetcdf 4.9.2 in conda-forge/libnetcdf-feedstock#175. Would it be useful to have it for 4.9.1 as well? The backport would be easy enough, but it's always a little bit of a hassle to release older versions. Would pinning to !=4.9.1 be an option?

Thanks @zklaus!

That's definitely a sensible option.

We will take a look next week. Getting a lot of conflicts with different PR's updating dependencies today.

[trexfeathers]

I've tested this locally and it's looking good! In a meeting now so will push up in an hour or so.