Can I intentionally produce a pure Python non-platform wheel?

[zanecodes]

Description

I understand per #255 why it may be undesirable for cibuildwheel to produce a pure-Python wheel since cibuildwheel has no a priori way to determine if this was intentional, so it must assume it was not; a reasonable assumption to make.

However, my use case involves wrapping cibuildwheel in a larger GitHub workflow for reuse within our organization, and we would prefer that the workflow abstract away the specifics of how the wheel is being built regardless of whether it is pure Python or platform specific, and avoid having to set some sort of build_platform_wheel: true property on a per-repository basis.

I would really appreciate having the ability to pass an option to cibuildwheel so that it does not throw this error when this happens, and just builds the pure-Python wheel instead.

The only other workarounds I can envision are:

• Force our developers to explicitly specify if they want to build a platform wheel and only use cibuildwheel if they do (breaches the abstraction, prevents us from using identical workflow files across repositories hampering maintainability)
• Catch this specific error message and build the wheel again using python -m pip wheel (not easy to do within a GitHub Action, wastes time building twice)
• Somehow try to detect if the repository contains a C extension beforehand (parsing the setup.py options? grepping for *.c and *.cpp files? very brittle) and only use cibuildwheel if it does
• Somehow monkey patch cibuildwheel to make this error a no-op (not even sure this is possible)

If this is already possible some other way I haven't thought of, I'd be quite grateful to know what it is.
Thanks!

Build log

https://github.com/zanecodes/tweakwcs/runs/3745689759?check_suite_focus=true

CI config

https://github.com/spacetelescope/action-publish_to_pypi/blob/master/build-wheel/action.yml

[henryiii]

The workflows should be very different, though? In one case, you should run a single job that calls pipx run build to build your wheel and SDist, probably on linux for speed. In the other case, you want to build at least one job per supported platform, and unless the build is very fast, probably split it up some other way too, like per-arch or per-python version. You were worried about wasting time building twice - this would waste time on multiple unneeded jobs for all pure-Python builds. By the time you get to cibuildwheel, it's too late, you are already wasting time launching many jobs, especially limited macOS ones. You'd have horrible reporting in the checks, etc. And you'd be building the exact same wheel every time.

Why not have a reusable workflow (https://docs.github.com/en/actions/learn-github-actions/reusing-workflows) or composite action (https://docs.github.com/en/actions/creating-actions/creating-a-composite-action), and either use a setting or just ask users to select the pure Python or built one? I don't think it's that hard to ask someone who is able to write C extensions to select a different workflow option.

[henryiii]

See, for example, the workflows (not reusable, at least yet, just examples) at: https://scikit-hep.org/developer/gha_pure and https://scikit-hep.org/developer/gha_wheels. Those can't be merged into a single workflow even if cibuildwheel didn't throw an error - they are completely different long before cibuildwheel runs.

[zanecodes]

The workflows should be very different, though? In one case, you should run a single job that calls pipx run build to build your wheel and SDist, probably on linux for speed. In the other case, you want to build at least one job per supported platform, and unless the build is very fast, probably split it up some other way too, like per-arch or per-python version.

That's a very good point and something I somehow failed to consider, haha. We currently do per-platform and source distribution builds in parallel so the wasted time would be cut down a bit thanks to that, but multiple builds does seem like a great reason not to do this.

Why not have a reusable workflow (https://docs.github.com/en/actions/learn-github-actions/reusing-workflows) or composite action (https://docs.github.com/en/actions/creating-actions/creating-a-composite-action), and either use a setting or just ask users to select the pure Python or built one? I don't think it's that hard to ask someone who is able to write C extensions to select a different workflow option.

Since reusable workflows only went into public beta this week, we've unfortunately not been able to take advantage of them yet, but that is very much something I plan to do in the future. We do currently use a composite action though.

As much as I'd like to avoid adding a mandatory setting to our composite action, I think you're probably correct that it's the best option. Reusable workflows might save us here though, alleviating our maintainability concerns and making this option a bit more palatable.

Thanks for responding so quickly, I think reusable workflows and an optional setting would be a satisfactory solution.