New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't call host tools that are written in Python and use system packages #596
Comments
How is this a meson-python issue? If your analysis is correct, there isn't anything that meson-python can do about it. The default pip behavior is to build the package in an isolated Python environment, thus what you observe is pretty much by design. You can disable this with the |
It maybe isn't. I just wasn't sure where to file this otherwise. Would you agree that pip could be smarter about only isolating things it owns? Feel free to close this. (both solutions aren't possible, |
I'm not sure there is a way for pip to be smarter. I assume things are done the way they are for a reason.
Disabling build isolation requires to take care of the build dependencies, indeed, but I don't see alternatives. |
ok, thanks Let's hope no toolchain starts using Python in the future :P |
I think your use case and your concerns are very valid. But you would need to bring them up with the pip developers to see whether there is a possible solution. Dependencies on system tools and library are impossible or at least very difficult to manage in Python packages. @rgommers is doing some work in this area. |
In case meson-python is invoked via pip, any Python tools it uses can't access the system Python packages, which breaks g-ir-scanner, which we use for building the tests. Fix by not building tests when building a wheel. See mesonbuild/meson-python#596 Fixes #622
In theory, build isolation could use an actual venv with a python executable in the subprocess PATH. I am guessing (but don't know for sure) that they opted to implement build isolation using PYTHONPATH because it's a bit more lightweight? |
In case meson-python is invoked via pip, any Python tools it uses can't access the system Python packages, which breaks g-ir-scanner, which we use for building the tests. Fix by not building tests when building a wheel. See mesonbuild/meson-python#596 Fixes #622
IIRC |
I can confirm that pypa/build works: FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install --no-install-recommends -y \
python3-dev \
python3-pip \
libcairo-dev \
gobject-introspection \
libgirepository-1.0-dev \
gir1.2-glib-2.0 \
gir1.2-freedesktop-dev \
git
RUN git clone https://gitlab.gnome.org/GNOME/pygobject && cd pygobject && git checkout 3.48.0
RUN pip install --break-system-packages build==1.1.1
# broken:
# RUN cd pygobject && pip wheel .
# works:
RUN cd pygobject && pyproject-build . |
Upstream issue https://gitlab.gnome.org/GNOME/pygobject/-/issues/622
First of, we are going to remove this usage upstream, so not that big of a problem right now.
Problem (from what I understand):
import('gnome').generate_gir()
generate_gir()
internally calls/usr/bin/g-ir-scanner
which has a shebang of/usr/bin/python3
sitecustomize.py
which prevents any imports of system site-packages of all Python things called during the build/usr/bin/g-ir-scanner
can't import its dependency modules and failsPossible solution:
The only idea I could come up with is that pip itself would need to detect when the interpreter used isn't the one of the isolated build env (in sitecustomize.py) and reverts its restrictions and unsets the pythonpath: https://github.com/pypa/pip/blob/7881c5303c9afa152bb9938a2751e3df4f1077bc/src/pip/_internal/build_env.py#L97-L156
The text was updated successfully, but these errors were encountered: