Experiemental build env backend by pypa/build

[uranusjr]

The current build backend is moved to a subpackage, and the new one can be enabled with --use-feature=build.

[pradyunsg]

Oh fun, I was working on getting #10720 working locally, with the logic moved behind a flag. I guess I should push that so that we can share each other's logic and pick the best parts. :)

[uranusjr]

This one’s partly done as an exercise so I can have a better idea how build’s environment population logic fits into pip. And my conclusion after finishing it is… not very well. It’s doable, but the amount of glue code doesn’t seem worthwhile. I’m currently leaning toward pip should still create the build environment with its own logic (your PR), and run build on top of that instead.

[pradyunsg]

Hmm... I see OverlayBuildEnvironment which is basically mirroring the existing overlay / normal separation.

I do think we can/should get rid of that separation, since it's not particularly meaningful IMO -- and that's something I think we should do as a part of this change.

run build on top of that instead.

I'm not sure that there's not much value in that -- as far as I understand, build is basically environment isolation + a call to pep517 (which will soon become pyproject-hooks). At that point, we might as well call the hooks ourselves instead of doing that via build.

[uranusjr]

In that case it’s probably best if we don’t go through build at all. build’s environment creation is also only a thin layer on virtualenv and venv that pip can easily replicate.

[pradyunsg]

I do think there'd be value in moving to an implementation that aligns our virtual environment creation with build's default w/ venv, and having those two use the exact-same underlying code (i.e. moving that code into build, and using it from there).