Migrate setup.py to pyproject.toml

[dlstadther]

Problem

The use of installing a python project with python setup.py install is deprecated and should be replaced with alternative installation methods. The Luigi project still utilizes setup.py only.

Resolution Proposal

Luigi should be updated to replace its setup.py with a pyproject.toml. At the same time, I would suggest utilizing poetry or uv as a dependency management and build tool (either tool can use setuptools or hatch for building).

Related Issue(s)

One of the motivations behind #3326 was to aid in local verification of this change.

[hirosassa]

Can I catch this issue?

[dlstadther]

@hirosassa , if you're asking to work on this, feel free. I started on it when I had some free time, but ran into other commitments and haven't been able to spend the time on this (nor #3326 ) like I hoped.

Feel free to take anything i've already done. I'll include a link to my initial code changes (not complete) and my notes related to my plan.

---

Code: master...dlstadther:luigi:migrate-setup-to-pyproject

Notes:
Challenges/Questions

• how to grab version from meta.py?
• maintain access to luigi.version - Add internal version info #2760
• don't break pypi data - Fix package descriptions for pypi. #2958
• don't break tests
• ensure release-process doc is up-to-date
• how to test pypi uploads?
• how to test readthedocs?

Currently used keys of setup.py

• name - pypi name
• version - version tag (str)
• description - short description
• long_description - readme + extra detail
• author - pypi author(s)
• url - defaults to pypi "Homepage" url
• license - license type
• packages - list of packages in your project, including their subpackages, etc.
• package_data - additional files to include in distribution, typically data files
• entry_points - named callable a user might need to use
  • console_scripts - point to a function which will be made available as a CLI tool
• install_requires - minimal required packages which will be installed when pip installing
• extra_require - dict mapping of installable "extras" which are only installed when needed or requested
• classifiers - metadata for users (not used by pip)

Supporting files:

• README.rst - used for long-description key
• MANIFEST.in - includes additional files in distribution
• LICENSE.txt - used for license key

Special things that happen "today":

• subset of static files for package_data based on file extension
• conditional install_requires packages/versions based on OS (windows needs a different version of python-daemon)
• conditional install_requires packages/version based on ENV (readthedocs avoids installing python-daemon at all)

If using uv, the following commands could be helpful for the release process:

# uv build
uv build sdist

# uv publish (assumes token set at UV_PUBLISH_TOKEN)
uv publish

# incrementing versions
uvx hatch version patch
uvx hatch version minor
uvx hatch version major
uvx hatch version minor,rc