Revert "Use modern tomllib/tomli modules for reading TOML files"

[umohnani8]

This reverts commit c5a356f.

Reverting as there is packaging issues with the new toml library that was being added.

Signed-off-by: Urvashi Mohnani umohnani@redhat.com

[umohnani8]

@rhatdan @lsm5 PTAL

[rhatdan]

LGTM

[rhatdan]

@jwhonce @mwhahaha PTAL

[lsm5]

NOTE: tomli isn't included yet on RHEL. Since we're both the upstream as well as the RHEL downstream maintainers, any dependency on tomli is going to increase our overall maintenance costs, and also the toml package is still being worked on going by commit dates. We can and should revisit this if RHEL makes the switch to tomli.

/lgtm
/hold

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lsm5, umohnani8

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [lsm5,umohnani8]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[lsm5]

/hold cancel

[mgorny]

For the record, I would like to say that I find this greatly disappointing. It's ignoring the needs of the larger Python community in the name of your own convenience, and I feel that it was done completely without my back, without any attempt to communicate with people who actually need tomli.

That said, this probably needs podman-py is going to leave Gentoo sooner than later. We don't find software "maintained" when two trivial commits are made over 2.5 years, and important bugs are not receiving any attention.

[gotmax23]

Nack. Reverting a commit from another contributor with just two lines of justification and without involving them in the discussion is a no go in my book.

On the technical merits of the change: can't you keep the fallback and patch the setup.cfg on RHEL to depend on toml instead of tomli? Also, python3-toml is deprecated in Fedora; pushing an update that adds a new dependency on a deprecated() package is not allowed.

[hroncok]

One of the RHEL Python maintainers here. Talk to us please. I'd gladly maintain tomli in old RHEL for you if it's such a higher maintenence cost for you, rather than maintain upstream-dead pytoml or toml in new RHEL. Please don't make decisions in a vacuum. @mgorny's contribution was valuable to us and now we treat them like this, shame on us.

[lsm5]

I removed the hold too soon, so that's on me. My apologies.

One of the RHEL Python maintainers here. Talk to us please. I'd gladly maintain tomli in old RHEL for you if it's such a higher maintenence cost for you, rather than maintain upstream-dead pytoml or toml in new RHEL. Please don't make decisions in a vacuum. @mgorny's contribution was valuable to us and now we treat them like this, shame on us.

tomli is not there in any current RHEL, not just old rhel. It's only on epel. We did check with RHEL decision makers about this. It was their advice to revert it to unblock things. Also, I have contacted python-maint way back in the past about owning some other dependency for RHEL, and the response I got was: "You need it, you own it", which was also echoed again this time by said decision makers.

So if you / python-maint are yourselves committing to owning tomli on RHEL proper, then I'm fine with undoing this but I will leave it to @umohnani8 @rhatdan @jnovy. If they agree, it shouldn't take long to fix this.

[lsm5]

If team decides not to undo this revert and keep toml, then I will carry a patch in the fedora package for compliance.

[hroncok]

Let me know which RHEL versions @ which Python versions you need tomli on and I'll carry on (assuming said Python versions are still supported there). In the meantime, I kindly ask you to patch the package in RHEL to use the dead only-supported-in-RHEL toml libraries instead of patching the package in Fedora to use the current stuff, as that approach makes it easier for other downstream packagers.

[lsm5]

Let me know which RHEL versions @ which Python versions you need tomli on and I'll carry on (assuming said Python versions are still supported there). In the meantime, I kindly ask you to patch the package in RHEL to use the dead only-supported-in-RHEL toml libraries instead of patching the package in Fedora to use the current stuff, as that approach makes it easier for other downstream packagers.

Works for me. We need tomli on RHEL 8 and 9 and of course anything that comes after. @jnovy can probably confirm the exact .Y numbers and maybe also the python versions. I'm personally happy to carry the patch in RHEL until tomli gets in, but again, I'll let @jnovy confirm since he's the point person on the RHEL side.

Since @umohnani8 is the primary maintainer upstream, I'll let her make the final call. We can open a PR soon after.

[jnovy]

@hroncok Thanks for the initiative here Miro. I currently reverted the retrospective tomli dependency in downstream (RHEL) so that python-podman is at least buildable. I can drop this once python-tomli is present in RHEL. Let me know if you need a hand/sponsor for the python-tomli RHEL package review.

I'm not expert on python dependencies but one possible way forward would be to fallback to python-toml if python-tomli isn't found? Like that we don't need to forcibly introduce python-tomli in RHEL (unless that was already a plan) and still make python-podman work there? Thank you @lsm5 for looking into this too.

[hroncok]

Let me know if you need a hand/sponsor for the python-tomli RHEL package review.

Thanks for the offer, but I don't. I just need to know on which Python versions you need tomli in RHEL 8 and 9.

I'm not expert on python dependencies but one possible way forward would be to fallback to python-toml if python-tomli isn't found? Like that we don't need to forcibly introduce python-tomli in RHEL (unless that was already a plan) and still make python-podman work there?

Exactly. If you prefer runtime if conditionals or nested try-elses on runtime over a RHEL-only patch, you could do this:

1. try using tomllib, this is the best option but it is only available on Python 3.11+ (in RHEL 10, 9, 8)
2. fallback to tomli, which has the same API as tomllib, but is not (yet) available in RHEL 9+8 Python 3.9, 3.8, or 3.6
3. fallback to toml, which is dead upstream but is supported in RHEL 9+8 on Python 3.9
4. fallback to pytoml, which is dead upstream but is supported in RHEL 8 on Python 3.6 and RHEL 7 on Python 2.7

Note that toml/pytoml operates on text, while tomli/tomlib operates on bytes, so they have different API.

See for example micropipenv, which does something similar: https://github.com/thoth-station/micropipenv/blob/v1.4.5/micropipenv.py#L186

[rhatdan]

SGTM
I am hearing conflicts on which upstreams are dead or not. I would like to have python-podman use the latest, most supported libraries possible in the upstream and then fall back for RHEL.

We need community help in python-podman since some of the core maintainers have moved onto other projects. Any help for @umohnani8 would be appreciated.

[hroncok]

I am hearing conflicts on which upstreams are dead or not.

• pytoml is officially declared dead. The description on PyPI says "The pytoml project is no longer being actively maintained. Consider using the toml package instead." It implements outdated TOML 0.4. It is deprecated in Fedora and unwanted in ELN/RHEL 9+.

• toml isn't declared dead but certainly smells funny. It has indeed received 2 trivial commits recently, but the latest version was released 2 years ago and the project does not even declare support for Python greater than 3.9. It implements outdated TOML 0.5. It is deprecated in Fedora and unwanted in ELN/RHEL 10+.

• tomli is the same code as tomllib (± some Python < 3.11 compatibility shims and typing annotations). Both implement the current TOML standard version 1.0. tomllib is part of the Python 3.11+ standard library and has an active primary maintainer (author of tomli) and a secondary backup maintainer who is a core CPython contributor and also a member of the Red Hat Python Maint team. Unlike the above, it only implements reading/parsing TOML and hence has a much smaller codebase to maintain. tomli is unwanted in ELN/RHEL 10+ because it is not required on Python 3.11+.

[gotmax23]

As the maintainer of tomli in EPEL, I'd be more than happy to see it in RHEL 8 and 9. In the meantime, I can put together a patch to add a fallback to pytoml.

I just need to know on which Python versions you need tomli in RHEL 8 and 9.

I think it's just the default pythons (3.6 on 8 and 3.9 on 9).

[jnovy]

Yes, default pythons - python-3.6 for RHEL8 and python-3.9 in RHEL9 please. @gotmax23

[gotmax23]

I've submitted #244 to restore compatibility with RHEL

[umohnani8]

This SGTM also - if we can get help from python maintainers to support/own the latest libraries in RHEL, that will be really appreciated.

I am more than happy to revert this revert once we have support for the libraries and/or a way to fallback for RHEL.

Thank you for the help @gotmax23 @hroncok!

[hroncok]

• RHEL 9 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2175213
• RHEL 8 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2175215