CI: Bump to cibuildwheel 2.2.2, Python 3.10 wheels will now build and deploy

[bocklund]

No description provided.

[codecov]

Codecov Report

Merging #374 (3a3cd1c) into develop (c9607c8) will increase coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #374      +/-   ##
===========================================
+ Coverage    89.92%   89.95%   +0.02%     
===========================================
  Files           44       44              
  Lines         4368     4369       +1     
===========================================
+ Hits          3928     3930       +2     
+ Misses         440      439       -1     

Impacted Files	Coverage Δ
pycalphad/io/tdb.py	87.64% <0.00%> (+0.02%)	⬆️
pycalphad/core/starting_point.py	92.85% <0.00%> (+2.38%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9607c8...3a3cd1c. Read the comment docs.

[bocklund]

Currently failing because SciPy (a build dependency) wheels for 3.10 are not available and the builds from source are failing. Once a SciPy wheel is available, our wheels should build successfully. scipy/scipy#14567 is tracking upstream.

[bocklund]

Update

SciPy is waiting until Python 3.10 wheels are ready on all platforms to release the wheels on PyPI (per scipy/scipy#14567 (comment)). Linux wheels are ready and can be found in the Anaconda.org nightly SciPy wheels files, but Mac and Windows wheels are not ready yet. The blocker for SciPy's Mac/Windows wheels (this PR: MacPython/scipy-wheels#132) are NumPy wheels for Mac/Windows (this PR: MacPython/numpy-wheels#135).

[bocklund]

This needed a symengine bump to get the Python 3.10 wheels.

The Python 3.10 Test builds work using all the available wheels. For some reason, cibuildwheel isn't picking up the scipy 1.7.2 wheels on Linux and Windows, but macos seems to work fine.

The documentation build is also failing due to some Furo error and looks unrelated to this PR. I opened a discussion here.

[bocklund]

It looks like cibuildwheel is always trying to build scipy from source. On earlier versions of Python (3.8 and 3.9) scipy 1.7.2 source builds fail, but and pip falls back to an older wheel (scipy 1.7.1) successfully. For 3.10, wheels of older scipy builds are not available.

It seems like the question is why the scipy 1.7.2 wheels are not getting picked up correctly.

[bocklund]

Maybe it could be an architecture flag thing? The current scipy build tags which do not get picked up correctly by cibuildwheel look like:

scipy-1.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
scipy-1.7.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

while the older wheels that are picked up correctly look like:

scipy-1.7.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl

[richardotis]

Could it be that the version of pip is too old? See:

• https://www.python.org/dev/peps/pep-0600/
• Implement PEP-600: Future 'manylinux' Platform Tags for Portable Linux Built Distributions pypa/pip#9077

[bocklund]

I found something similar here:

pypa/cibuildwheel#633 (comment)

Per the wheel spec, the platform tag shall be split by .. Meaning the wheel will have: Tag: manylinux_2_5_x86_64 Tag: manylinux1_x86_64 Tag: manylinux_2_17_x86_64 Tag: manylinux2014_x86_64 If any one is valid, it can be installed. Older pip will disregard manylinux_2_5_x86_64, manylinux_2_17_x86_64 & manylinux2014_x86_64 because it doesn't know them. It will consider manylinux1_x86_64 valid and thus install the wheel.

That was from April 3, 2021. Our cibuildwheel builds seems like it's using pip 21.2.X, while the CI test builds use pip 21.3.X. Both should have this sorted out, since they were released more recently than that.

[bocklund]

https://www.python.org/dev/peps/pep-0599/ also related.

It seems like manylinux1 is older than manylinux2010 is older than manylinux2014. If my reading is correct, a manylinux2010 host is compatible with manylinux1 wheel, but a manylinux2014 wheel is not compatible with a manylinux2010 host. From above, the scipy 1.7.1 has a manylinux1 wheel, but 1.7.2 is manylinux2014 and manylinux_2_17. (https://github.com/pypa/manylinux for more details on tags).

It may be that we need to update our cibuildwheel platform since the docker image looks to be for manylinux2010.

[richardotis]

It may be that we need to update our cibuildwheel platform since the docker image looks to be for manylinux2010.

I bumped cibuildwheel to 2.2.2 (latest version) in a recent commit and it didn't seem to help. Shouldn't the Docker images update with each release?

[bocklund]

Not necessarily. The defaults for the images are still manylinux2010 (see https://cibuildwheel.readthedocs.io/en/stable/options/#linux-image).

I tried to bump to 2014 just to see what happens.

[bocklund]

Looks better on x86-64. Note that scipy 3.10 manylinux wheels are only on x86_64 and aarch64 platforms, so i686 are expected to fail.

It is possible to configure cibuildwheel to skip/selectively enable platforms for different python versions. For sure it works in the TOML via overrides (see https://cibuildwheel.readthedocs.io/en/stable/options/#overrides). I'm not sure about configuring overrides via YAML, but you can have both active (with YAML taking priority).

[bocklund]

Not sure what the long term plan is upstream (numpy/scipy) for i686 support, so maybe disabling them for all python versions is the future anyway. i686 could be dropped along with win-32.

[richardotis]

musllinux builds should be skipped, right?

[bocklund]

Correct. Looks like musllinux support was added in cibuildwheel 2.2.0

[bocklund]

🎉 🎉 🎉

Okay, so in summary:

• Bump cibuildwheel to 2.2.2
• Add Python 3.10 support for building wheels and running tests
• Don't build musilinux wheels (consistent with previous behavior)
• Explicitly drop 32bit support
• Drop support for manylinux2010 wheels in favor of manylinux2014 wheels across all Python versions
• Use oldest-supported-numpy as a build and development requirement

Do all these seem reasonable @richardotis?

[richardotis]

Yes, please merge.