Generate requirements files from pyproject.toml

[mwtoews]

This PR is based on numpy/numpydoc#483 which makes pyproject.toml a primary file without any template, and adds a pre-commit script to write requirements files. Some of the comments from the requirements files are moved to pyproject.toml.

[mwtoews]

Question: why is there a "default" extra? It repeats the same info in the dependencies. The script can easily be modified to generate this file too, if that extra were removed.

[lagru]

Hey @mwtoews thanks for the suggestion! It seems useful to me to have the source of truth all originate from one place. Especially because it may make it easier to use comments in the pyproject.toml. Before I review, I am curious, did the previous discussion in numpydoc bring up any other pros or cons?

The default.txt is used in several places in the CI to install the runtime dependencies in a controlled manner independent of the package itself e.g. see this search. I am not sure if there is a clear advantage to doing it that way but for now would hope to keep the noise to our infrastructure minimal.

[mwtoews]

Before I review, I am curious, did the previous discussion in numpydoc bring up any other pros or cons?

Most of the discussion was from a superseded PR numpy/numpydoc#481 where I was aiming to use a dynamic tool-dependant feature of setuptools to read optional dependencies (rather than using a script and template approach). Ultimately, there are more strengths in having a single statically-analyzable pyproject.toml (complete with all dependencies), but some tools also need to pip install -r with individual requirements files too. The applied PR for numpydoc and in this PR is to also remove the template approach, since requirements files don't really need a template (except for a comment header). A clear benefit is that edits to pyproject.toml now directly go to that file, and where present, editors may enable syntax highlighting (template files can't do that).

As for the requirements/default.txt file, I have not problems with it. But in pyproject.toml it is duplicated. From pip's perspective pip install scikit-image and pip install scikit-image[default] do the same thing. My guess is removing the default extra and generating a requirements files from [project.dependencies] will not change anything, and resolve the duplication.

[lagru]

Thanks for addressing my suggestions! I have only a minor one left.

[lagru]

Thanks for addressing all my concerns so quickly! I tweaked the hook configuration a bit in 8d8bf24 to make it more specific otherwise ready to merge IMO.