pip install h5py fails on building wheels for numpy, with and without numpy already installed

[jpmvferreira]

Setting up a clean virtual environment and trying to install h5py fails with the following error:

    ----------------------------------------
    ERROR: Failed building wheel for numpy
  Failed to build numpy
  ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
  ----------------------------------------

This has happened in two different machines, when installing with a clean virtual environment running python version 3.10 and numpy 1.21.4.

The full tracelog when executing pip install h5py --verbose with numpy already installed with pip install numpy on a virtual environment with gcc and g++ also installed conda-forge, the same source as the Python binary, is as follows:
https://pastebin.com/WP2gdGtX

The same happened with the default gcc and g++ shipped by Manjaro and on a machine running CentOS 6.6 with both compilers also installed from conda-forge.

This also happens if I pip install h5py right after I install Python from conda-forge, on a clean virtual environment.

This is odd behaviour as I already had a virtual environment running h5py, also version 3.5.0, with numpy 1.21.3 and now, when replicating the same virtual environment which I saved, I get the same error (at least the end of it, this thing is huge) followed by:

ERROR: Could not find a version that satisfies the requirement h5py==3.5.0 (from versions: 2.2.1, 2.3.0b1, 2.3.0, 2.3.1, 2.4.0b1, 2.4.0, 2.5.0, 2.6.0, 2.7.0rc2, 2.7.0, 2.7.1, 2.8.0rc1, 2.8.0, 2.9.0rc1, 2.9.0, 2.10.0, 3.0.0rc1, 3.0.0, 3.1.0, 3.2.0, 3.2.1, 3.3.0, 3.4.0, 3.5.0)
ERROR: No matching distribution found for h5py==3.5.0

Am I missing something obvious here?

One quick side-note, this virtual environment was created using micromamba, not conda. However the first aims to be fully compatible with the latter and so far there were no compatability issues reported so far. Also this issue happened a few days ago, I had already had a virtual environment where h5py was sucessfully installed without issues.

The previous is probably relevant because I've tried installing h5py using micromamba and got the following:

$ micromamba install -c conda-forge h5py

                                           __
          __  ______ ___  ____ _____ ___  / /_  ____ _
         / / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
        / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
       / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
      /_/

conda-forge/linux-64     Using cache
conda-forge/noarch       Using cache

Pinned packages:
  - python 3.10.*

Encountered problems while solving:
  - nothing provides hdf5 1.8.15* needed by h5py-2.6.0-np110py27_3

ERROR   Could not solve for environment specs

Which is either a packaging issue or a problem with micromamba in fetching dependencies. Either way it's not relevant to the previous bug but I decided to share it regardless.

System information:

• Operating System: Manjaro
• Python version: 3.10
• Where Python was acquired: Conda-forge
• h5py version: latest
• HDF5 version: latest

[takluyver]

Do you need to use Python 3.10? We haven't made a release with wheels for 3.10 yet, so you'll have an easier time if you can use Python 3.9. It looks like conda-forge also doesn't have Python 3.10 packages (for some reason the command you show is trying to install an ancient version of h5py for Python 2.7).

If you do need to use Python 3.10, you'll be building h5py from source. By default, that tries to install a particular version of NumPy in a build environment. You can use pip's --no-build-isolation flag to avoid that - you may find you need to install some extra packages to build h5py in that case.

[jpmvferreira]

Okay so that's the piece of the puzzle I was missing, kinda obvious once I think about it, and testing with python 3.9.7 works as expected, thanks.

Next time how should one figure out that's the source of the issue by himself, to avoid opening up an issue? I've browsed through the documentation, but since there were no warnings or incompatibilities that I saw applying, I assumed it was a bug that was yet to be found.

[takluyver]

I don't really have a good answer for that. There's just always a delay between a new version of Python coming out and compiled packages like h5py publishing pre-built packages for it. People are working to reduce that - we're meant to be able to build with a Python release candidate before the final release. But in our case, we're dependent on Azure pipelines (our main CI service) adding the new version of Python, which typically takes a couple of weeks after the final release, and on NumPy publishing wheels first.

It will hopefully work a bit more gracefully next time - we're switching (#1993) to use oldest-supported-numpy. That should mean that when a new Python comes along, the fallback is to build with the newest NumPy instead of the same version we use for the previous Python version.

[jpmvferreira]

Okay thanks for information.