feat: Add PIP_OPTIONS CMake option for greater control of Python bindings install

[matthewfeickert]

• Resolves Add CMake flag PIP_OPTIONS to allow for passing specific options to pip during the install #1646
  • Addresses root cause of Issue python:: local installation of python client/bindings #1641, but won't close it until the next release with this PR in it.
• Closes python: add env var for install customizations #1647

This PR will:

• Add CMake option PIP_OPTIONS to allow the user to pass in pip options at build configuration time to customize the Python bindings install. As --verbose is a pip option, this makes the PIP_VERBOSE CMake option added in PR fix: Install Python bindings with pip if available #1586 redundant, and so it is removed. Additionally, as --prefix is also a pip option, append the selected default value of $ENV{DESTDIR}/{CMAKE_INSTALL_PREFIX} as a --prefix pip option if there is no --prefix given by the user. If there is a given --prefix from the user in PIP_OPTIONS already, warn the user that this might not be a good idea unless they know what they're doing.

• Remove PYTHON_LAYOUT from override_dh_auto_configure as not used in Python 3 builds.

I would personally be in favor of this option over PR #1647 as it makes things explicit at build time and so much more transparent for debugging, comparisons, and reproducibility.

[matthewfeickert]

@simonmichal This should be ready for review (assuming the rpm-centos7 CI passes).

@adriansev If you can take a look at this as well that'd be great. In future releases you should definitely be able to have things working in a way that solves Issue #1641 -DPIP_OPTIONS="--ignore-installed" but you might also be able to have things work by just not using the PIP_OPTIONS flag at all, given the removal of --force-reinstall by default in this PR.

[adriansev]

@matthewfeickert so, i checkout out the PR and gave it a spin:

1. without any additional options the python bindings are not installed and i get:

Processing ./bindings/python
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
ERROR: Could not find a version that satisfies the requirement OFF (from versions: none)
ERROR: No matching distribution found for OFF

i suspect that some defaults (empty variable) are not well enough processed

2. when specifying -DPIP_OPTIONS="--ignore-installed" everything just worked, so YUPII :)

[matthewfeickert]

@adriansev Can you try again without any additional options for the Python bindings? I think the correction @henryiii provided will fix things for you now (e.g. PIP_OPTIONS is now properly a STRING by default and not a BOOL if no input is passed).

Though as he notes, please clean your cache first if you're building locally (e.g. cmake --build <build dir> --clean-first)

[adriansev]

@matthewfeickert tried again without any customization on my side, and seems to work like expected, meaning that the wheel was created, my user side xrootd (.local/lib/python3.10/site-packages) was removed (as expected i think without ignored-installed) then the new wheel was installed in the defined private location. From my side i would say that everything works and it's ready to be merged. Thanks a lot for all your work!

[matthewfeickert]

From my side i would say that everything works and it's ready to be merged.

Fantastic! Hopefully this is the last time you and ALICE get hit with some strange behavior from XRootD Python bindings. :)

@simonmichal As @adriansev has verified things are working for the problems raised in Issue #1641 this is ready for review. 👍 Let me know if anything isn't clear!

[simonmichal]

@matthewfeickert : again, thanks a lot for all your work :-)

[simonmichal]

P. S. it seems there's a glitch on GitHub, I'll merge it later on today.

[matthewfeickert]

P. S. it seems there's a glitch on GitHub, I'll merge it later on today.

Yeah, GitHub basically was down for everyone today. 🤪 One of those days I guess and probably a tough one for GitHub engineering.