silx.io: Changed open function behavior: lock HDF5 files when opening them

[t20100]

Checklist:

• The PR title is formatted as: <Module or Topic>: <Action> <Summary> (see contributing guidelines)

---

This PR reverts PR #3939 which disabled HDF5 file locking when opening a file with silx.io.open.
Disabling HDF5 file locking is causing trouble (see #4071) because there is an issue with external links when file locking is disabled (see https://forum.hdfgroup.org/t/external-link-and-file-locking-disabled-issue/12012).
So as of today, disabling file locking when opening a HDf5 file is not safe and IMO shouldn't be the default.

Looking at the Larch link in the issues PR #3939 solved(see issues #3938, #3936), it looks it is calling directly h5py_utils.File instead of silx.io.open for hdf5 file so it should not be affected.

attn @woutdenolf @maurov

[maurov]

@t20100 thanks for making me aware of this change. In Larch, the wanted default behavior is to have file locking disabled by default because we want to be able to read HDF5 files while their are open in BLISS for writing, without blocking or corrupting the file. My understanding is that silx.io.h5py_utils.File is the entry point for this behavior. Do you confirm it?

[t20100]

My understanding is that silx.io.h5py_utils.File is the entry point for this behavior. Do you confirm it?

Yes, I confirm. You can also use h5py.File(... locking=False) with recent versions of h5py and libhdf5 (with h5py_utils.File you don't have to care about the versions).

However beware that we recently found an issue with external links when disabling file locking: If you open a hdf5 file and access an external link from it, the "external" file is opened with file locking. This is causing issues which motivates reverting the change in silx.io.open.

[maurov]

However beware that we recently found an issue with external links when disabling file locking: If you open a hdf5 file and access an external link from it, the "external" file is opened with file locking. This is causing issues which motivates reverting the change in silx.io.open.

Thanks for letting me know about this bug. Please, could you tell me how to programmatically recognize that a group in a HDF5 file is an external link?

[t20100]

You can use h5py.Group.get and h5py.ExternalLink:

isinstance(h5group.get("/h5/path", getlink=True), h5py.ExternalLink)
# or
h5group.get("/h5/path", getclass=True, getlink=True) is h5py.ExternalLink

But beware (again!), this only tells you whether or not the last part of the path is an external link.
For instance, "/ext_link_group/data" will not be reported as an external link even if "/ext_link_group" is an external link to a group... You need to check each part of the path.