Staged installation of Python bindings not working after migration to pip

[amadio]

Original bug report from Gentoo Linux: https://bugs.gentoo.org/861452

Bisection with git points to commit 31e3d71 as the problematic change.

How to reproduce the problem:

$ # Configure with Python enabled and install prefix in the system (e.g. /usr)
$ cmake -S $PWD -B build -DENABLE_PYTHON=1 -DPYTHON_EXECUTABLE=/usr/bin/python3 -DCMAKE_INSTALL_PREFIX=/usr
$ cmake --build build
$ # Perform a staged installation into a DESTDIR which is not empty (e.g. to a directory install in the current directory):
$ env DESTDIR=$PWD/install cmake --build build --target install

What I get:

... (many removed lines of output)
-- Processing: /home/amadio/src/xrootd/install//usr/share/man/man1/xrdcp.1
-- Processing: /home/amadio/src/xrootd/install//usr/share/man/man1/xrdmapc.1
-- Installing: /home/amadio/src/xrootd/install/usr/lib64/libXrdClHttp-5.so
-- Set runtime path of "/home/amadio/src/xrootd/install/usr/lib64/libXrdClHttp-5.so" to ""
Processing ./bindings/python
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: xrootd
  Building wheel for xrootd (setup.py): started
  Building wheel for xrootd (setup.py): finished with status 'done'
  Created wheel for xrootd: filename=xrootd-2022.822+dc680e5f8-cp310-cp310-linux_x86_64.whl size=711034 sha256=ca7888c135d15b360c8f8df1f7c05222855e3bf39bd7e0105ced715581f59d67
  Stored in directory: /tmp/pip-ephem-wheel-cache-k0enpw1z/wheels/4f/bf/c3/60b496a0b3d3cc1bbe87053fb501e707645f2f8833a13d0c18
Successfully built xrootd
Installing collected packages: xrootd
  Attempting uninstall: xrootd
    Found existing installation: xrootd 5.4.3
    Uninstalling xrootd-5.4.3:
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/usr/lib/python3.10/site-packages/XRootD/__init__.py'
Consider using the `--user` option or check the permissions.

Edit (09/11/2022): There are some existing issues with pip for this behavior: pypa/pip#3063, pypa/pip#1716, and pypa/pip#2677.

[matthewfeickert]

@amadio You should just use the PIP_OPTIONS CMake option that was added in PR #1648 in your cmake build if you truly need --ignore-installed.

Example:

xrootd/.github/workflows/build.yml

Lines 136 to 142 in dc680e5

	cmake3 \
	-DCMAKE_INSTALL_PREFIX=/usr/local/ \
	-DPYTHON_EXECUTABLE=$(command -v python3) \
	-DENABLE_TESTS=ON \
	-DPIP_OPTIONS="--verbose --force-reinstall --prefix /usr/local/" \
	-S xrootd \
	-B build

Though I'm skeptical of why --ignore-installed is needed still (c.f. #1769 (comment)). Regardless, can you verify that this option works for the problem you reported?

[abh3]

Shall we close this then?

[amadio]

Yes, I think we can close this.

[amadio]

I've decided to take a look at this again due to #1807 and because I think that make DESTDIR=$INSTALL install should work by default without having to change $PIP_OPTIONS. The patch below seems sufficient (just add the same --prefix option from the command used on Debian):

--- a/bindings/python/CMakeLists.txt
+++ b/bindings/python/CMakeLists.txt
@@ -95,7 +95,7 @@ else()
     CODE
     "EXECUTE_PROCESS(
       COMMAND /usr/bin/env ${XROOTD_PYBUILD_ENV} ${PYTHON_EXECUTABLE} -m pip install \
-                ${PIP_OPTIONS} \
+                --prefix \$ENV{DESTDIR}/${CMAKE_INSTALL_PREFIX} ${PIP_OPTIONS} \
                 ${CMAKE_CURRENT_BINARY_DIR}
       )"
   )

After applying this, with the default CMAKE_INSTALL_PREFIX=/usr/local I now see things get installed in the right place with a staged installation:

build $ tree -d install
install
└── usr
    └── local
        ├── bin
        ├── include
        │   └── xrootd
        │       ├── XProtocol
        │       ├── Xrd
        │       ├── XrdCl
        │       ├── XrdHttp
        │       ├── XrdNet
        │       ├── XrdOuc
        │       ├── XrdPosix
        │       ├── XrdSec
        │       ├── XrdSys
        │       ├── XrdXml
        │       ├── XrdXrootd
        │       └── private
        │           ├── Xrd
        │           ├── XrdCl
        │           ├── XrdNet
        │           ├── XrdOuc
        │           ├── XrdPosix
        │           ├── XrdSecsss
        │           ├── XrdSys
        │           └── XrdZip
        ├── lib
        │   └── python3.10
        │       └── site-packages
        │           ├── XRootD
        │           │   ├── __pycache__
        │           │   └── client
        │           │       └── __pycache__
        │           ├── pyxrootd
        │           │   └── __pycache__
        │           └── xrootd-2022.1020+3a0998333.dist-info
        ├── lib64
        └── share
            ├── man
            │   ├── man1
            │   └── man8
            └── xrootd
                └── cmake

42 directories

@matthewfeickert What do you think about this?

[StefanBruens]

Using "--prefix" for DESTDIR is wrong, as pip may then write the DESTDIR into some generated files. The correct way to specify DESTDIR is by using "--root". E.g.:
--root "\$ENV{DESTDIR}" ${PIP_OPTIONS}

Using --root "" should have the same effect as omitting the parameter, i.e. this should also work when DESTDIR is not set.

[amadio]

Unfortunately, using --root doesn't prevent pip from trying to uninstall stuff from /usr, which is what we want to prevent when doing staged installations (which are not installing into /usr, but into a staging area). This is described in pypa/pip#3063, which is also linked in the description.

epsftws build $ python3.10 -m pip install --root $PWD/install --prefix /usr bindings/python
Processing ./bindings/python
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: xrootd
  Building wheel for xrootd (setup.py) ... done
  Created wheel for xrootd: filename=xrootd-5.5.3.post196-cp310-cp310-linux_x86_64.whl size=740187 sha256=6db2dd81ffc1f1b10d29e4aa4e299644e2c4d7084fa44302675b158c82094738
  Stored in directory: /tmp/pip-ephem-wheel-cache-y388wzs6/wheels/4f/bf/c3/60b496a0b3d3cc1bbe87053fb501e707645f2f8833a13d0c18
Successfully built xrootd
Installing collected packages: xrootd
  Attempting uninstall: xrootd
    Found existing installation: xrootd 5.5.3
    Uninstalling xrootd-5.5.3:
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/usr/lib/python3.10/site-packages/XRootD/__init__.py'
Consider using the `--user` option or check the permissions.