Isolated build environment is corrupted on Homebrew Python 3.10 without active virtual environment #11539
Comments
RazerM
added
S: needs triage
|
FWIW |
|
How did you install the Python 3.10.8? The assert checks that a path is not in |
|
brew |
|
I'm hitting the same issue. This happens only with Python 3.9 and 3.10 installed via Homebrew. It works on earlier Python versions. I'm debugging the issue but the first actionable thing could be to make the assertion error more verbose and detailed: understanding that something is going wrong with how pip setups the build venv took me many hours of head scratching. Mostly for reference to other maybe bumping into the same problem, the symptoms look like this: where the interesting part is: Setting |
|
You might want to reach out to maintainers of Homebrew’s Python formula to get a better idea about the issue. It only happens to their Python installation so it’s likely something wrong with how they build or configurate it. |
|
The The assertion in there obviously fails. |
|
Quoting myself above:
The assertion is doing what it is supposed to do, those entries being in there is the problem. |
|
If I read it correctly, the first part of the generated |
|
Oh, and by the way, adding a thing twice to a list while checking that it is not already there, rarely succeeds. The last three lines of that Deduplicating the list solves the problem. |
|
The logic works everywhere except Homebrew’s Python installation, some I’m inclined to say the responsibility to correct this likely lies in Homebrew. pip can modify the heuristic to better support whatever Homebrew is doing, if we can have insight first on what Homebrew is doing with its Python installation (that nobody else is), why it’s doing it, whether it’s a correct thing (if it’s not, maybe fix that instead), and how pip can help without compromising what we (both sides) are trying to achieve. |
uranusjr
added
project: <downstream>
|
Following the code that generates pip's I'm trying to understand why this works as expected in a virtual environment. |
|
@uranusjr Maybe I'm off the wrong track, but shouldn't pip pass Changing this fixes the issue for me: diff --git a/src/pip/_internal/locations/_sysconfig.py b/src/pip/_internal/locations/_sysconfig.py
index 0bbc9283d..e1071006a 100644
--- a/src/pip/_internal/locations/_sysconfig.py
+++ b/src/pip/_internal/locations/_sysconfig.py
@@ -214,5 +214,5 @@ def get_platlib() -> str:
def get_prefixed_libs(prefix: str) -> typing.Tuple[str, str]:
- paths = sysconfig.get_paths(vars={"base": prefix, "platbase": prefix})
+ paths = sysconfig.get_paths(vars={"base": prefix, "platbase": prefix}, scheme="venv")
return (paths["purelib"], paths["platlib"])With Homebrew's Python the default installation scheme (outside a virtualenv) is 'osx_framework_library': {
'stdlib': '{installed_base}/{platlibdir}/python{py_version_short}',
'platstdlib': '{platbase}/{platlibdir}/python{py_version_short}',
'purelib': '/usr/local/lib/python{py_version_short}/site-packages',
'platlib': '/usr/local/{platlibdir}/python{py_version_short}/site-packages',
'include': '{installed_base}/include/python{py_version_short}{abiflags}',
'platinclude': '{installed_platbase}/include/python{py_version_short}{abiflags}',
'scripts': '/usr/local/bin',
'data': '/usr/local',
}, |
|
Judging from the prefix you showed ( Also, the |
|
Homebrew has the |
|
But the scheme is not available anywhere else (on 3.10) so we can’t do that. But since that would be available in 3.11, we can adopt it conditionally, and “incidentally” resolve the issue for Homebrew. Would you be able to file a pull request for this? We can work out the details from there. |
|
Sure. Let me double check that this solves my CI issues and then I'll come up with a PR. |
|
Great Scott! This fixes Python 3.10 but not 3.11. Edit: it was an operator error. The patch above solves the problem for both Python 3.10 and 3.11. I'm preparing a PR. |
get_prefixed_libs() computes the Python path for libraries in a pip
isolation environment. Python 3.11 introduced the "venv" path scheme
to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the
default paths scheme when Python is invoked outside a virtual
environment is "osx_framework_library" and does not relative to the
"{base}" or "{platbase}" variables.
Fixes pypa#11539.
get_prefixed_libs() computes the Python path for libraries in a pip
isolation environment. Python 3.11 introduced the "venv" path scheme
to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the
default paths scheme when Python is invoked outside a virtual
environment is "osx_framework_library" and does not relative to the
"{base}" or "{platbase}" variables.
Fixes pypa#11539.
get_prefixed_libs() computes the Python path for libraries in a pip
isolation environment. Python 3.11 introduced the "venv" path scheme
to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the
default paths scheme when Python is invoked outside a virtual
environment is "osx_framework_library" and does not relative to the
"{base}" or "{platbase}" variables.
Fixes pypa#11539.
get_prefixed_libs() computes the Python path for libraries in a pip
isolation environment. Python 3.11 introduced the "venv" path scheme
to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the
default paths scheme when Python is invoked outside a virtual
environment is "osx_framework_library" and does not relative to the
"{base}" or "{platbase}" variables.
Fixes pypa#11539.
get_prefixed_libs() computes the Python path for libraries in a pip
isolation environment. Python 3.11 introduced the "venv" path scheme
to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the
default paths scheme when Python is invoked outside a virtual
environment is "osx_framework_library" and does not relative to the
"{base}" or "{platbase}" variables.
Fixes pypa#11539.
uranusjr
changed the title
get_prefixed_libs() computes the Python path for libraries in a pip
isolation environment. Python 3.11 introduced the "venv" path scheme
to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the
default paths scheme when Python is invoked outside a virtual
environment is "osx_framework_library" and does not relative to the
"{base}" or "{platbase}" variables.
Fixes pypa#11539.
Description
I experienced the following error in a CI environment
Since I know from the
pip install -vlog above this truncated output thatskbuildwas definitely picked up from pyproject.toml and installed, it's clear that the assertion error prevents theskbuildpackage from being found.Enabling
PYTHONVERBOSE=1:I see that it comes from
pip/src/pip/_internal/build_env.py
Line 130 in ff207cf
This is not my project so I don't know if this is something wrong with the package itself, the CI environment where this happens, or something else.
Changing the CI job to use a venv gets rid of the issue (which is of course a good idea anyway), but it would be good to understand why this assertion is in place and what I should do to fix it.
I think the message should at least include
pathandsys.path, which I could open a PR for, but I would also like some context from someone familiar with the implementation about what one should do in this case.Expected behavior
The AssertionError would include actionable information
pip version
22.3
Python version
3.10.8
OS
macOS 10.14.6
How to Reproduce
I don't yet know why this happens, and since this issue about improving the message I think it's ok.
Output
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: