PEP 517 and build isolation are disabled in Gradle plugin pip

[mhsmith]

We're currently using pip 19.2.3, which has full PEP 517 support. But it currently fails with an error similar to pypa/pip#9953, apparently caused by pip being unable to find its own modules within a ZIP:

            pip {
                install "black==21.5b0"
                options "--no-binary", "black"
            }

Collecting black==21.5b0
  Using cached https://files.pythonhosted.org/packages/52/7e/6a41d1504225ef9213731e69ecebfa8fe78c614bcb01e1147f6242073636/black-21.5b0.tar.gz
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\smith\AppData\Local\Programs\Python\Python38\python.exe' 'C:\Users\smith\cygwin\git\chaquo\python\server\pypi\pkgtest\app\build\generated\python\bp.zip\pip\_vendor\pep517\_in_process.py' prepare_metadata_for_build_wheel 'C:\Users\smith\AppData\Local\Temp\tmpfva405pg'
         cwd: C:\Users\smith\AppData\Local\Temp\pip-install-s8m_9tdu\black
    Complete output (1 lines):
    C:\Users\smith\AppData\Local\Programs\Python\Python38\python.exe: can't find '__main__' module in 'C:\\Users\\smith\\cygwin\\git\\chaquo\\python\\server\\pypi\\pkgtest\\app\\build\\generated\\python\\bp.zip\\pip\\_vendor\\pep517\\_in_process.py'
    ----------------------------------------
ERROR: Command errored out with exit status 1: 

If we add --no-use-pep517, we instead get:

ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml

So I've added --no-use-pep517 to the default options, because it gives a clearer error message. And I've patched pip/_internal/pyproject.py so it falls back on setup.py even if a pyproject.toml file is present.

Of the top 1000 packages on PyPI, the only ones which currently fail because of this are:

• pendulum (poetry build system): has a pure-Python fallback which currently fails to work.
• orjson (maturin build system for Rust)

They now produce the following message:

ERROR: Disabling PEP 517 processing is invalid: project does not have a setup.py

I've had no requests for either of these packages yet, so this isn't urgent. There may also be other packages which use alternative build systems, but as long as they provide compatible wheels, that wouldn't affect us in this context.

[mhsmith]

We also use the --no-build-isolation option option to prevent pip from installing the build-time requirements specified by PEP 518. I started to fix this in 9c98e40, but gave up when I realized it would also require altering build_env.py to keep our modified copy of setuptools on the PYTHONPATH. I then backed out these changes entirely when updating to pip 19.2.3.

Again, I'm not aware of any significant packages which fail because of this option, because any package which is modern enough to have a pyproject.toml file is probably also modern enough to be releasing wheels.

Since pip implements build isolation by calling itself in a subprocess in the new environment, we'll probably encounter the same ZIP-related error as above.

It's also likely that projects will request a specific version of setuptools to give them a more reproducible build. So if we enable build isolation in the future, providing a pre-patched copy of setuptools wouldn't be good enough anymore: we'd need to monkey-patch it at runtime with the same PYTHONPATH and sitecustomize technique used by build-wheel:

• Our setuptools modifications are fairly limited (currently only setuptools/monkey.py and pkg_resources/__init__.py), so it probably wouldn't be hard to support a wide range of setuptools versions this way. And because PYTHONPATH is inherited by subprocesses, there would be no need to patch build_env.py as mentioned above.
• BeeWare has already implemented this for iOS: beeware/Python-Apple-support@92089b7

[mhsmith]

This could probably be fixed by unpacking the ZIP before running pip. This will add a small delay to the first build of each project, but it shouldn't be too bad.

On Windows there may be a risk of exceeding the filename length limit, but we've already encountered and worked around a similar limit in zipimport, so we'll probably get away with it.