Improve packaging [PEP 517 + 621] #7076

cdce8p · 2022-06-27T13:26:21Z

Description

Use isolated build environments PEP 517 with python -m build instead of deprecated python setup.py sdist bdist_wheel
Move project metadata from setup.cfg to pyproject.toml PEP 621
Cleanup and remove MANIFEST.in -> entries haven't been used.
~~Add README.rst to MANIFEST.in. Required to be able to build the wheel from sdist and still include the readme text in the project metadata.~~
Remove setup.py. With recent pip versions (>= 21.0), no longer required for editable installs. setup.cfg is enough.

Metadata diff

-Author: Python Code Quality Authority
-Author-email: code-quality@python.org
+Author-email: Python Code Quality Authority <code-quality@python.org>
...
-Keywords: static code analysis linter python lint
+Keywords: static code analysis,linter,python,lint

Closes #4193

* PEP 517 * Include readme in sdist to be able to build wheel from it * Update release ci job to use build

coveralls · 2022-06-27T14:20:49Z

Pull Request Test Coverage Report for Build 2576530244

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 95.324%

Totals
Change from base Build 2571502189:	0.0%
Covered Lines:	16655
Relevant Lines:	17472

💛 - Coveralls

Pierre-Sassoulas

Thank you ! I wanted to wait for pep660 but what's left of the setup.cfg is really manageable in the end. This would close #4193

MANIFEST.in

Pierre-Sassoulas · 2022-06-27T15:20:32Z

Is there no change in the contributor doc at all ?

cdce8p · 2022-06-27T16:02:38Z

We can probably remove this entirely now ? Doesn't the pyproject.toml handles it ?

I first checked with just ´[build-system]when I got the error. You're correct though, with the[project]table, it works withoutMANIFEST.in`.

Is there no change in the contributor doc at all ?

pip install . and pip install -e . continue to function as expected. The main change is in the way the release package is build. python -m build instead of python setup.py sdist bdist_wheel but that is handled by Github actions anyway.

DanielNoord

Let's not remove setup.py just yet:
dependabot/dependabot-core#4483

This is quite an annoying bug.

This reverts commit 7798d91.

cdce8p · 2022-06-27T23:59:00Z

Update

Everything took a bit longer then I expected, but it's ready now.
I've also opened pylint-dev/astroid#1670 to update astroid, too.

README.rst and MANIFEST.in

Turns out if you remove the MANIFEST.in completely, README.rst will be included automatically. That's why the wheel build didn't fail. I've added README.rst explicitly now, just in case we do add some other files there in the future.

Let's not remove setup.py just yet:
dependabot/dependabot-core#4483

This is quite an annoying bug.

Thanks @DanielNoord! Didn't know about that. I've restored setup.py and added a comment.

Pierre-Sassoulas

Perfect ! We've come a long way from the 2000's packaging !

DanielNoord · 2022-06-28T06:51:14Z

setup.cfg

@@ -93,6 +28,11 @@ known_third_party = platformdirs, astroid, sphinx, isort, pytest, mccabe, six, t
 skip_glob = tests/functional/**,tests/input/**,tests/extensions/data/**,tests/regrtest_data/**,tests/data/**,astroid/**,venv/**
 src_paths = pylint

+[flake8]


Can't this go in .flake8?

Let's do it the other way round and move the whole flake8 config into setup.cfg. In the past, we have also removed .isort. At some point we can move most / all of these configs into pyproject.toml.

Strong agree. I don't think .flake8 support pyproject.toml right now though . But there's other tool we could move already, tox and black probably.

(PyCQA/flake8#234 (comment))

Strong agree. I don't think .flake8 support pyproject.toml right now though . But there's other tool we could move already, tox and black probably.

I would recommend to keep tox (and pylint) separate. Both have quite complex configurations. It's much easier to have them in their individual files. We could however move isort, pytest, and mypy.

cdce8p · 2022-06-28T13:23:05Z

@DanielNoord dependabot/dependabot-core#4483 doesn't seem to be an issue anymore. I just checked one of my repos and dependabot runs fine even without a setup.py file. Let's try it. We can always re-add it if causes issues.

github-actions · 2022-06-28T13:57:30Z

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit d380aca

DanielNoord · 2022-06-28T14:58:21Z

Are you sure? These are the logs/statuses for pydocstringformatter, which also doesn't have a setup.py.
https://github.com/DanielNoord/pydocstringformatter

cdce8p · 2022-06-28T16:34:16Z

Are you sure? These are the logs/statuses for pydocstringformatter, which also doesn't have a setup.py. https://github.com/DanielNoord/pydocstringformatter

Yeah! This is from python-typing-updater. Dependabot just opened a new PR for black with the last run a few hours ago. Not exactly sure, why it's not working for you though. Wild guess, maybe it's because you don't have a project table in your pyproject.toml? https://github.com/cdce8p/python-typing-update

My suggestion would be to just try it without setup.py. If it fails, I can open a PR to add it back.

DanielNoord · 2022-06-28T21:25:11Z

Let's do that. I think @Pierre-Sassoulas csn report whether the check is running successfully afterwards!

Addressed feedback

cdce8p · 2022-06-28T21:45:56Z

Seems like you were right @DanielNoord 😞 Triggered the run manually just now.
I'll open a PR shortly to add setup.py back.

That just raises the question: Why does it work in my repo https://github.com/cdce8p/python-typing-update?
A difference I could think of is the requirements.txt file. Both pylint and pydocstringformatter do only have requirements*<...>.txt files.

DanielNoord · 2022-06-28T21:47:33Z

@cdce8p I have no idea really... The annoying thing is that in de dependabot issue there is already a solution. But my complete lack of ruby experience makes it impossible to contribute any test so I hesitated to create a PR...

cdce8p · 2022-06-28T21:52:29Z

Followup PR: #7082

DanielNoord · 2022-07-31T16:58:51Z

Followup PR: #7249.

cdce8p added 4 commits June 27, 2022 14:17

Use isolated build environments

2842e4f

* PEP 517 * Include readme in sdist to be able to build wheel from it * Update release ci job to use build

Use new project metadata format [PEP 621]

74ae6dd

Cleanup unused MANIFEST entries

abd9179

Update misc files

c196772

This comment has been minimized.

Sign in to view

Pierre-Sassoulas added the Maintenance Discussion or action around maintaining pylint or the dev workflow label Jun 27, 2022

Pierre-Sassoulas reviewed Jun 27, 2022

View reviewed changes

MANIFEST.in Show resolved Hide resolved

cdce8p added 2 commits June 27, 2022 17:45

Remove MANIFEST.in

7798d91

Fix flake8

47b8c90

Fix keywords

366d05b

This comment has been minimized.

Sign in to view

DanielNoord requested changes Jun 27, 2022

View reviewed changes

cdce8p mentioned this pull request Jun 27, 2022

Improve packaging [PEP 517 + 621] pylint-dev/astroid#1670

Merged

cdce8p added 2 commits June 28, 2022 01:34

Revert "Remove MANIFEST.in"

3edc9f0

This reverts commit 7798d91.

Keep setup.py

ff074bf

cdce8p force-pushed the improve-packaging branch from a6590db to ff074bf Compare June 27, 2022 23:39

Add dependabot to custom dictionary

b089d4a

This comment has been minimized.

Sign in to view

Pierre-Sassoulas approved these changes Jun 28, 2022

View reviewed changes

DanielNoord previously requested changes Jun 28, 2022

View reviewed changes

Move remaining flake8 config to setup.cfg

e047d47

This comment has been minimized.

Sign in to view

cdce8p requested a review from DanielNoord June 28, 2022 12:27

Remove setup.py

e860338

Update pre-commit config

d380aca

cdce8p merged commit 0d36a63 into pylint-dev:main Jun 28, 2022

cdce8p deleted the improve-packaging branch June 28, 2022 21:39

cdce8p added this to the 2.15.0 milestone Jun 28, 2022

cdce8p mentioned this pull request Jun 28, 2022

Add setup.py back to fix dependabot #7082

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve packaging [PEP 517 + 621] #7076

Improve packaging [PEP 517 + 621] #7076

cdce8p commented Jun 27, 2022 •

edited

Loading

This comment has been minimized.

coveralls commented Jun 27, 2022 •

edited

Loading

Pierre-Sassoulas left a comment

Pierre-Sassoulas commented Jun 27, 2022

cdce8p commented Jun 27, 2022

This comment has been minimized.

This comment has been minimized.

DanielNoord left a comment

cdce8p commented Jun 27, 2022

This comment has been minimized.

Pierre-Sassoulas left a comment

DanielNoord Jun 28, 2022

cdce8p Jun 28, 2022

Pierre-Sassoulas Jun 28, 2022 •

edited

Loading

Pierre-Sassoulas Jun 28, 2022

cdce8p Jun 28, 2022

This comment has been minimized.

cdce8p commented Jun 28, 2022

github-actions bot commented Jun 28, 2022

DanielNoord commented Jun 28, 2022

cdce8p commented Jun 28, 2022

DanielNoord commented Jun 28, 2022

cdce8p commented Jun 28, 2022

DanielNoord commented Jun 28, 2022

cdce8p commented Jun 28, 2022

DanielNoord commented Jul 31, 2022

Improve packaging [PEP 517 + 621] #7076

Improve packaging [PEP 517 + 621] #7076

Conversation

cdce8p commented Jun 27, 2022 • edited Loading

Description

This comment has been minimized.

coveralls commented Jun 27, 2022 • edited Loading

Pull Request Test Coverage Report for Build 2576530244

💛 - Coveralls

Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

Pierre-Sassoulas commented Jun 27, 2022

cdce8p commented Jun 27, 2022

This comment has been minimized.

This comment has been minimized.

DanielNoord left a comment

Choose a reason for hiding this comment

cdce8p commented Jun 27, 2022

This comment has been minimized.

Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

DanielNoord Jun 28, 2022

Choose a reason for hiding this comment

cdce8p Jun 28, 2022

Choose a reason for hiding this comment

Pierre-Sassoulas Jun 28, 2022 • edited Loading

Choose a reason for hiding this comment

Pierre-Sassoulas Jun 28, 2022

Choose a reason for hiding this comment

cdce8p Jun 28, 2022

Choose a reason for hiding this comment

This comment has been minimized.

cdce8p commented Jun 28, 2022

github-actions bot commented Jun 28, 2022

DanielNoord commented Jun 28, 2022

cdce8p commented Jun 28, 2022

DanielNoord commented Jun 28, 2022

cdce8p commented Jun 28, 2022

DanielNoord commented Jun 28, 2022

cdce8p commented Jun 28, 2022

DanielNoord commented Jul 31, 2022

cdce8p commented Jun 27, 2022 •

edited

Loading

coveralls commented Jun 27, 2022 •

edited

Loading

Pierre-Sassoulas Jun 28, 2022 •

edited

Loading