Repository navigation
fix(macos): make matching python3-config available in build/test venvs - #2922
Conversation
Virtualenvs do not contain a python3-config (or versioned
python{X.Y}-config) script, and cibuildwheel only puts the venv bin
directory on PATH. The python.org framework builds ship these scripts
next to the base interpreter, but that directory is never added to PATH,
so C/C++ extensions that invoke python3-config picked up an unrelated
(system/Homebrew) python3-config, producing broken or ABI-mismatched
wheels.
After creating the venv on POSIX platforms, symlink the base
interpreter's python*-config scripts into the venv bin so the correct
one is found first on PATH. Interpreters without such a script (PyPy,
GraalPy, some uv-managed builds) are handled gracefully and never fail
the build.
Assisted-by: ClaudeCode:claude-opus-4.8
Fixes ruff (ANN/TC) and mypy (no-untyped-def) failures in CI. Assisted-by: ClaudeCode:claude-opus-4.8
4b8d3a5 to
968ce48
Compare
Refactor _symlink_python_config_scripts docstring to clarify behavior regarding linking python-config scripts.
|
if we aim to fix this, should a symlink be added or do we want to add the base python bin folder to the PATH instead (right after the venv one) ? |
|
If pypy3-config and graalpy3-config do exist (based on the AI description at least) I think we should similarly match for them and include them for consistency, otherwise it just becomes yet another case where our builds across interpreters are different from each other (and perhaps something a later AI review will flag). |
Nice idea! It feels a bit less hacky than modding the venv IMO. |
Per review, expose the base interpreter's bin directory on PATH right after the venv's, rather than symlinking python*-config scripts into the venv. This also covers pypy3-config/graalpy3-config and anything else shipped next to the base interpreter, and applies to both build and test venvs. Assisted-by: ClaudeCode:claude-fable-5
|
🤖 AI text below 🤖 Switched to the PATH approach suggested above: |
|
Could I get a review? Does a thumbs up count? :) |
agriyakhetarpal
left a comment
agriyakhetarpal
left a comment
There was a problem hiding this comment.
Sorry about holding off on this; this LGTM – it's just been conference season and I haven't been around lately :)
4 participants
🤖 AI text below 🤖
Problem
On macOS, cibuildwheel builds and tests inside a virtualenv and only adds the venv's
bindirectory toPATH. Virtualenvs don't containpython3-config, so builds that invoke it (for include dirs / link flags) pick up an unrelated one from elsewhere onPATH— typically the system or Homebrew Python — producing broken wheels or ABI mismatches.Fix
As suggested in review, the base interpreter's
bindirectory is added toPATHright after the venv's (POSIX only), for both build and test venvs. The matchingpython3-config/python3.X-configis then found first, and the same holds for anything else shipped next to the base interpreter (e.g.pypy3-config,graalpy3-config). The venv itself is not modified, and itsbinstill comes first, sopython/pipcontinue to resolve to the venv.Callers that need stricter isolation are unaffected: iOS rewrites
PATHwholesale after venv creation, and Linux builds don't use this code path.Closes #2021