Adapt distutils3 pattern for new packaging standards

[phmccarty]

The Python community has developed new packaging standards in the last few years, and some PyPI packages have already adopted them and cannot be updated until autospec supports these new standards. Over the coming years, we will continue to see more projects convert to the new standardized layout and remove backwards compatibility with distutils-based and setuptools-centric approaches.

As a starting point to see how the new system works (at a basic level), see https://packaging.python.org/tutorials/packaging-projects/.

This blog post summarizes the new BKMs pretty well: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

Other references:
https://www.python.org/dev/peps/pep-0517/
https://www.python.org/dev/peps/pep-0518/
https://www.python.org/dev/peps/pep-0621/

[insilications]

Yes... So how would CLR proceed with respect to its modifications to unixccompiler.py when building setuptools-based packages? Creating additional .avx* shared libraries and trying to link them is a very nice feature. Isn't the problem isolated only to packages that use a different a strictly different build system like poetry?

[phmccarty]

The main structural issue is that autospec's handling of pypi packages centers around the existence of a setup.py file at toplevel, so that baseline assumption will need to be revisited to accommodate the new standards. We will also be able to improve pypi dependency detection, since the new approaches use a more declarative approach that will be easier to parse.

(Another improvement will be for autospec to prefer parsing setup.cfg for setuptools-based projects instead of scraping setup.py to detect any dependency metadata. That's not strictly related to the standards I linked above, but that would be an improvement for our current distutils3 pattern.)

[bryteise]

With #695 we have added the initial pyproject support. The change keeps the existing distutils3 pattern wholly intact.