Import h5py fails due to undefined symbol

[MarDiehl]

• I read the conda-forge documentation and could not find the solution for my problem there because this error happens when building a package.

Issue:
Rebuild of a Python package fails in the test stage because h5py cannot be imported.
The relevant part of the log file on windows is

  File "D:\bld\damask-pkgs_1635640207683\_test_env\lib\site-packages\damask\util.py", line 27, in <module>
    import h5py
  File "D:\bld\damask-pkgs_1635640207683\_test_env\lib\site-packages\h5py\__init__.py", line 33, in <module>
    from . import version
  File "D:\bld\damask-pkgs_1635640207683\_test_env\lib\site-packages\h5py\version.py", line 15, in <module>
    from . import h5 as _h5
  File "h5py\h5.pyx", line 1, in init h5py.h5
ImportError: DLL load failed while importing defs: The specified procedure could not be found.

and on Linux

  File "/home/conda/feedstock_root/build_artifacts/damask-pkgs_1635640089558/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.9/site-packages/damask/util.py", line 27, in <module>
    import h5py
  File "/home/conda/feedstock_root/build_artifacts/damask-pkgs_1635640089558/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.9/site-packages/h5py/__init__.py", line 33, in <module>
    from . import version
  File "/home/conda/feedstock_root/build_artifacts/damask-pkgs_1635640089558/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.9/site-packages/h5py/version.py", line 15, in <module>
    from . import h5 as _h5
  File "h5py/h5.pyx", line 1, in init h5py.h5
ImportError: /home/conda/feedstock_root/build_artifacts/damask-pkgs_1635640089558/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.9/site-packages/h5py/defs.cpython-39-x86_64-linux-gnu.so: undefined symbol: H5Pset_fapl_ros3

The full logs are available on https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_apis/build/builds/400217/logs/38 and https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_apis/build/builds/400217/logs/41

[MarDiehl]

note: This could be a duplicate of #97

[takluyver]

I think #92 describes the underlying cause of this and #97: the dependency specified between h5py and HDF5 is not specific enough, so conda will install versions that are not compatible.

As far as I'm concerned, h5py is only reliably ABI compatible with the build of HDF5 (version & build options) it was built against. Some other things may work, but I wouldn't assume they do.

[MarDiehl]

Is there the possibility to pin the version via a hash?

[tschaka1904]

@MarDiehl, did you manage to get it working?

[MarDiehl]

@MarDiehl, did you manage to get it working?

I seems to work now, I assume due to updates to h5py/hdf5.

[tschaka1904]

What version of h5py are you using @MarDiehl? Tried to look into your logs, but they seem to be gone.

I'm trying to get 3.1 or 3.2.1 working.

[MarDiehl]

What version of h5py are you using @MarDiehl? Tried to look into your logs, but they seem to be gone.

I'm trying to get 3.1 or 3.2.1 working.

I'm not using any version at the moment, I got the issue when building my package. You could fork https://github.com/conda-forge/damask-feedstock, do some arbitrary changes and create a pull request. That should trigger the CI pipeline which reports the versions.

[khamyl]

Have the same problem with

h5py 3.6.0
hdf5 1.10.6

[tschaka1904]

Locking h5py to 3.1.0 worked for me, in the end. Still, I’d rather order to be able to use the latest.

[khamyl]

h5py 3.1.0 nompi_py39hf27771d_100 conda-forge
hdf5 1.10.6 h7ebc959_0

is not working either :(

[takluyver]

If you want to use a newer h5py, try installing a newer hdf5 as well. 1.12.1 is the current latest version.

[khamyl]

h5py 3.6.0 nompi_py39hd4deaf1_100 conda-forge
hdf5 1.12.1 nompi_h2a0e4a3_103 conda-forge

Unfortunately neither this combination. Every time it is the same:

(tf-gpu) d:\WS\Python\Tests>python
Python 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import h5py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\z0049ntw\Miniconda3\envs\tf-gpu\lib\site-packages\h5py\__init__.py", line 33, in <module>
    from . import version
  File "C:\Users\z0049ntw\Miniconda3\envs\tf-gpu\lib\site-packages\h5py\version.py", line 15, in <module>
    from . import h5 as _h5
  File "h5py\h5.pyx", line 1, in init h5py.h5
ImportError: DLL load failed while importing defs: The specified procedure could not be found.

[khamyl]

It seems one solution should be:

• uninstalled both the h5py and hdf5
• pip install h5py (not conda install)

Now have only h5py | 3.6.0 | pypi_0 | pypi and it can be imported :) Interesting...

[takluyver]

The wheel package (what you get with pip) bundles libhdf5 into the wheel, so it should never 🤞 have a mismatch between the HDF5 used to build h5py and the HDF5 it finds at runtime. That approach has its downsides, though - e.g. the read-only S3 driver is enabled in conda-forge, but not in pip, because that would mean bundling more libraries.

[zjgt]

I encountered the same problem, took inspiration from this issue, especially from Khamyl, and solved it by following:

1. create a new conda environment with python=3.7
2. pip install h5py; at this step, import anndata as ad worked.
3. pip install scanpy python-igraph leidenalg; everything seems to work fine.
   It did not work when I create a conda environment without defining python 3.7. As it stands, my current working anndata version is 0.8.0.

[openSourcerer9000]

Any update on this?

in a py3.10 env i'm noticing I'm also getting these errors for fiona and rasterio:
ImportError: DLL load failed while importing _version: The specified procedure could not be found.

[batterseapower]

Pinning to h5py=3.1 is suboptimal because that version doesn't contain a bugfix (h5py/h5py#1931) that is necessary to work with the latest numpy release. So if you pin to h5py=3.1 you also need to do numpy<1.24

[dwr-psandhu]

with the lastest pull from conda-forge I get h5py 3.11 that fails with the same error.
pip install h5py --force-reinstall does the trick for me