Move from setup.py to pyproject.toml

[jakirkham]

Increasingly there is a move by Python projects to use pyproject.toml to declare their metadata and dependencies and away from setup.py. Raising this issue to start discussion around how to approach this in Dask.

xref: dask/distributed#7622

[jrbourbeau]

Thanks @jakirkham! This has come up before in the context of tool configuration (i.e. migrating setup.cfg to pyproject.toml) dask/distributed#5725. In general folks seem to be neutral to +1 for switching to pyproject.toml. I'm not very familiar with pyproject.toml vs. setup.py but my understanding is pyproject.toml seems to be the more modern version (maybe it's already considered a best practice? I'm not sure).

Given our current setup.py are you aware of any subtitles we'll need to account for in migrating to a pyproject.toml? Or is it pretty much a copy-and-paste, then tweak the formatting?

[jakirkham]

Think the main thing we need to be aware of is we are doing some dynamic things with optional requirements in dask, which would need to change.

With distributed, this isn't an issue.

[jrbourbeau]

Okay, but optional dependencies like pip install dask[dataframe] are still supported, right? Probably just need to do something like what pandas is doing here

[jakirkham]

Sorry should add that the main thing with pyproject.toml is it is declarative as opposed to setup.py, which opens the door to arbitrary code execution (and the usual warts with build dependency management). The Python community is trying to move to the declarative approach to alleviate these issues (related blogpost).

Since distutils is dropped from Python 3.12 ( PEP 632 ), this is moving a bit quicker.

Edit: GH didn't show the last comment. Yes it is possible to list out optional dependencies.