We should use an older setuptools as build-system

[mauritsvanrees]

Since setuptools 69.3, sdists get an underscore instead of a dot.
For example look on PyPI at the files of plone.app.linkintegrity 4.0.3 and 4.0.4.

This is a problem for Buildout: buildout/buildout#647

See also zestsoftware/zest.releaser@934ffb7, which is part of zestsoftware/zest.releaser#449 which influences this a bit, but does not help us.

In Plone we have this in most packages in pyproject.toml:

[build-system]
requires = ["setuptools>=68.2"]

I think we want it like this now:

[build-system]
requires = ["setuptools>=68.2,<=69.2"]

At least that makes sure that for new packages source dists can be installed by buildout, at least until the problem there is solved, which is hard.

Note that we always release wheels as well, and for pure Python packages like we have in Plone these are fine, and can be installed on all systems. For packages with C code, like several Zope ones, this is different.

[jensens]

Or switch to hatch? @ericof showed it to me at Beethovensprint and it looked very convincing.
But i am not sure what buildout says then...

[ericof]

I discussed, last week, with @sneridagh that we probably should use hatch for the Python packages and replace tox (used here) with it.
Actually, this is one of the selling points of hatch

Opinions @mauritsvanrees, @gforcada ?

[mauritsvanrees]

Do you mean replace tox with hatch? Or replace setuptools with hatchling? The last could be an alternative solution to the current issue, although I highly doubt it (I expect that hatchling will also generate the new normalised source distribution names that current Buildout cannot install). The first is a completely separate one.

I have solved the problems in Buildout though, pending review, see buildout/buildout#649. So that buys some time, at least until the next setuptools or pip release that changes something...

hatch and hatchling could certainly be interesting, but not something I personally want to dive into right now.

[ericof]

@mauritsvanrees Both options.

Replacing tox -> hatch

Considering our current usage of tox, we could easily replace it with hatch and reduce the complexity of our current setup.
Right now, we have the main package installation (used to run the instance) and another one to run tests, which makes debugging a bit convoluted experience (adding breakpoints and so on).
Also, everything we have in tox — a matrix of Python versions and different environments — is already covered by hatch and managed inside pyproject.toml
Another point for using hatch is that we can simplify our Makefiles, as hatch makes it easier to create 'script' commands like check, test, etc.

Replacing setuptools -> hatchling

Although it is not a must-have for our core packages, getting rid of setup.py, setup.cfg, and MANIFEST.in in favor of pyproject.toml is desirable in the long run.
Also, the fact zest.releaser works with hatch + pyproject.toml, makes the transition very, very smooth.

Summary

First of all, I'm bringing this here because I would like to have a similar setup for Python packages that are either:

• Core packages: plone.exportimport
• Backend addons: collective.mastodon
• Policy packages inside projects: 2024.ploneconf.org

Having the same toolset and commands to create (cookieplone-templates), develop, test locally, test in the ci, and then release them.

I'm willing to invest some time figuring this out, but I would then want to have it supported in plone/meta otherwise it is just another way of doing the same things :-)

[mauritsvanrees]

Replacing tox -> hatch

Yes, can be interesting. I just have no experience with hatch yet, so hard to say if it will help us.

Replacing setuptools -> hatchling

Although it is not a must-have for our core packages, getting rid of setup.py, setup.cfg, and MANIFEST.in in favor of pyproject.toml is desirable in the long run. Also, the fact zest.releaser works with hatch + pyproject.toml, makes the transition very, very smooth.

Long run: yes.

Short run: no. Unfortunately, as I saw yesterday, Buildout cannot develop pyproject.toml only packages. :-(
So for core Plone packages I would want to avoid it in Plone 6.

Having the same toolset and commands to create (cookieplone-templates), develop, test locally, test in the ci, and then release them.

Yes please.

I'm willing to invest some time figuring this out, but I would then want to have it supported in plone/meta otherwise it is just another way of doing the same things :-)

If it works: sure, we could switch.

[jensens]

So, reading all this: should we drop buildout-support in Plone 7? Maintaining our stack for buildout-support feels like riding with the handbrake on.

[mauritsvanrees]

So, reading all this: should we drop buildout-support in Plone 7? Maintaining our stack for buildout-support feels like riding with the handbrake on.

It starts sounding like it yes. :-/ I managed to fix some problems during the sprint, but more challenges creep up.

[gforcada]

Sorry for my long silence for the last few months, live is not easy as of late... 🙇🏾 🙏🏾

Big 👍🏾 on dropping buildout on Plone 7, or as early as possible.

Pinning to a specific setuptools version seems to be the quickest and least path of resistance for now, right?

hatch/hatchling: I have no stock options from tox, actually I'm quite a newbie, and anyway, all credits for plone/meta go to zopefoundation/meta. That's why we are using tox in plone/meta.

In my small flake8 plugins I maintain, at some point I switched them all to hatchling and after the usual struggles to get it right, I haven't had any problem so far 👍🏾

The link about hatch environment sounds nice, is there any proof of concept though? Small tool/projects are easy to migrate, but Plone has quite some quirks(?).

I might try it on my flake8 plugins during the summer, if time and energy allows.

The good thing of having plone/meta is that we can make any change on it and deploy it to all packages ✨

Though, changing too much the CLI of how things interact can provide quite some burden, so we better make first some testing and then communicate that plone core at X point is moving from tox to hatch and how the translation on the day to day use this means. We don't want to confuse our tooling and specially not our developers 😅

I haven't been following much the new cookiecutter-based project scaffolding projects, but what are they using now? hatch already? Makefile only? tox w/ or w/o plone/meta?

That part, together with the bobtemplates should be at some point (during the conference) be organized (it is already?) so that it is clear from all parts what's the way forward, and any users of all the tools know about it... if that's actually possible to agree on 🙄