Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: enable free-threaded wheel builds [wheel build] #26512

Merged
merged 1 commit into from
May 27, 2024

Conversation

ngoldbaum
Copy link
Member

I think all that's needed is this one-line change and the 3.13 wheel builder will automatically create a free-threaded wheel. Let's see...

@ngoldbaum ngoldbaum added 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703) component: distribution labels May 23, 2024
@ngoldbaum ngoldbaum force-pushed the free-threaded-wheels branch 7 times, most recently from 4248f66 to 98c4a07 Compare May 23, 2024 21:01
@ngoldbaum
Copy link
Member Author

Unfortunately I think we might need a cython release that builds numpy under free-threaded python. At least if we want to avoid hacks like the change I made to pyproject.toml, which I definitely don't think is a good idea.

I don't think there's any other way to customize the build requirements? Could I maybe do a sed replacement in pyproject.toml on the build image in the cibw_before_build.sh script?

@ngoldbaum

This comment was marked as outdated.

@ngoldbaum ngoldbaum force-pushed the free-threaded-wheels branch 2 times, most recently from 3842ff1 to 0994ab8 Compare May 23, 2024 21:40
@ngoldbaum
Copy link
Member Author

Looks like the wheels successfully built for manylinux and musllinux: https://github.com/numpy/numpy/actions/runs/9215227199

@rgommers rgommers self-requested a review May 24, 2024 10:15
Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is indeed not mergeable with the change to pyproject.toml. In addition, we may not want to have the cp313t builds in wheel.yml, because I don't think we want to upload them to PyPI for 2.1.0. So how about this:

  • copy wheels.yml with these changes to free_threaded.yml
  • edit that file so it only contains what is needed for free-threaded builds (no sdist, no other platforms, no upload to the STAGING bucket)
  • set the PIP_NO_BUILD_ISOLATION=1 env var
  • then build Cython from latest master (no change to pyproject.toml needed anymore, because of turning off build isolation)

Then we can merge this and start uploading to https://anaconda.org/scientific-python-nightly-wheels/numpy

@ngoldbaum ngoldbaum force-pushed the free-threaded-wheels branch 8 times, most recently from 5196394 to bb037ec Compare May 27, 2024 18:29
@ngoldbaum ngoldbaum marked this pull request as ready for review May 27, 2024 18:47
@ngoldbaum
Copy link
Member Author

Then we can merge this and start uploading to https://anaconda.org/scientific-python-nightly-wheels/numpy

I ended up doing this in a slightly different way because setting PIP_NO_BUILD_ISOLATION messes with cibuildwheel's assumptions for the build environment, but it works out to the same thing more or less.

The tests pass on both manylinux and musllinux with the GIL disabled - I think the wheels should be ready to upload along with the nightlies.

Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! All seems to work as advertised, nice work Nathan. Please feel free to merge once the comment cleanup is done.

.github/workflows/free-threaded-wheels.yml Outdated Show resolved Hide resolved
@ngoldbaum ngoldbaum force-pushed the free-threaded-wheels branch 2 times, most recently from 3081228 to 81e089f Compare May 27, 2024 19:49
PKG_CONFIG_PATH="$PWD/.openblas"
DYLD="$DYLD_LIBRARY_PATH:/$PWD/.openblas/lib"
echo "CIBW_ENVIRONMENT_MACOS=$CIBW PKG_CONFIG_PATH=$PKG_CONFIG_PATH DYLD_LIBRARY_PATH=$DYLD" >> "$GITHUB_ENV"
fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for anyone curious: I'm leaving the windows and mac-specific stuff in here even though it's unused because our intention is to add mac builds and then windows builds when that's possible.

[skip cirrus] [skip azp] [skip circle]
@rgommers rgommers added this to the 2.1.0 release milestone May 27, 2024
@ngoldbaum ngoldbaum merged commit 2cefb78 into numpy:main May 27, 2024
93 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703) component: CI component: distribution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants