Increase minimum supported Sphinx to 4.2

[AA-Turner]

Documentation

#108184 introduces a nice improvement to https://docs.python.org/dev/library/string.html by linking the tokens. However linking to other groups was only introduced in Sphinx 3.5 (sphinx-doc/sphinx#8247).

Using the same survey as #86986 (comment):

• Ubuntu 23.04: Python 3.11, Sphinx 5.3.0
• Debian stable: Python 3.11, Sphinx 5.3.0
• RHEL doesn't seem to list versions
• Fedora 38: Python 3.11, Sphinx 5.3.0
• OpenSUSE rolling: Sphinx 7.2.4

Updated survey: #109209 (comment)

A

Linked PRs

• GH-109209: Bump the minimum Sphinx version to 4.2 #109210
• [3.12] GH-109209: Bump the minimum Sphinx version to 4.2 (GH-109210) #109636
• [3.11] GH-109209: Bump the minimum Sphinx version to 4.2 (GH-109210) #109637

[AA-Turner]

Ahh. Sphinx 3.5.4 doesn't work on Python 3.10.1 and newer, as it tries to import types.Union: https://github.com/sphinx-doc/sphinx/pull/8803/files#diff-f41e19a9b77f4670d4dd1ea61f80d518c93a4403112fd7e91253dac47cf55fe5

The venv I was using locally had Python 3.9, so I was blissfully unaware.

From the survey in the issue, 4.2 (where this was fixed: https://github.com/sphinx-doc/sphinx/pull/9513/files#diff-f41e19a9b77f4670d4dd1ea61f80d518c93a4403112fd7e91253dac47cf55fe5) is also fine, so I'll update this PR.

The alternative would be to run the workflow on Python 3.9, if keeping Sphinx 3.5 would be preferred.

A

[serhiy-storchaka]

I'm waiting for 7.2 for the :no-typesetting: option. But it won't be soon.

[AA-Turner]

@doko42 @stefanor @mitya57 -- we're considering raising the minimum required Sphinx to a newer version. It seems that Debian stable has Python 3.11 and Sphinx 5.3 -- it would be useful to understand what would be useful/inconvenient to you as redistributors.

For example if we went all the way to Sphinx 7.2 (for the feature Serhiy mentioned), and backported that to Python 3.11 and 3.12, how painful would that be?

A

[AA-Turner]

@vstinner @hroncok @hrnciar @torsava -- as above, it would be useful to understand what would be useful/inconvenient to you from Fedora -- it seems that Fedora 38 has Python 3.11 and Sphinx 5.3.0.

From Fedora's perspective, if CPython went all the way to Sphinx 7.2 and backported that to Python 3.11 and 3.12, how painful would it be?

(Do you know if there are separate RHEL people to ask as well?)

A

[AA-Turner]

@marketayord @dirkmueller @danigm -- please may I ask the same questions as above for OpenSUSE?

A

[vstinner]

@vstinner @hroncok @hrnciar @torsava -- as above, it would be useful to understand what would be useful/inconvenient to you from Fedora -- it seems that Fedora 38 has Python 3.11 and Sphinx 5.3.0.

cc @befeleme

[hroncok]

@vstinner @hroncok @hrnciar @torsava -- as above, it would be useful to understand what would be useful/inconvenient to you from Fedora -- it seems that Fedora 38 has Python 3.11 and Sphinx 5.3.0.

• Fedora 40: 7.1.2 (we plan to update to 7.2 or newer eventually) @ Python 3.12
• Fedora 39: 6.2.1 @ Python 3.12
• Fedora 38: 5.3.0 @ Python 3.11
• Fedora 37: 5.0.2 @ Python 3.11 (end of life plan: 2023-11-14)

From Fedora's perspective, if CPython went all the way to Sphinx 7.2 and backported that to Python 3.11 and 3.12, how painful would it be?

Doing it in the main branch would be OK. Backporting that to 3.12 and 3.11 would mean we would no longer be able to build up-to-date documentation for Python in Fedora . We would need to come up with a solution. (One of the solutions is that we would keep the outdated documentation in Fedora, another is that we could build an alternate sphinx package in version 7.2 but that can still have an unforeseen impact on other packages...) It would not be the end of the world for us, but it would certainly not be convenient at all.

Do you know if there are separate RHEL people to ask as well?

The Python-Maint team at Red Hat (@encukou @vstinner @hrnciar @torsava @befeleme @stratakis @frenzymadness @hroncok) takes care of Python in Fedora Linux and RHEL both.

Nothing to worry about for RHEL, we don't build the documentation from the source there at all.

[mitya57]

Regarding Debian: we still have many packages which fail to build with Sphinx 7.x. See this list: https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=sphinx7.1;users=python-modules-team@lists.alioth.debian.org.

I want to give some more time for the maintainers to fix these bugs, but it's a few weeks, hopefully not months.

[danigm]

@marketayord @dirkmueller @danigm -- please may I ask the same questions as above for OpenSUSE?

From Fedora's perspective, if CPython went all the way to Sphinx 7.2 and backported that to Python 3.11 and 3.12, how painful would it be?

In Tumbleweed we just have sphinx 7.2.4 and use it to build python documentation for all supported versions (3.8, 3.9, 3.10, 3.11 and 3.12), so, as far as I know, it won't change anything there. In openSUSE Leap we've also Python 3.11 and Sphinx 4.2, so maybe that will require the inclusion of a new version of Sphinx there. But as far as I can tell, it won't be very painful.

Maybe @mcepl can add something else about the implication of this change in openSUSE or correct if I'm missing something.

[AA-Turner]

Also copying @mgorny from Gentoo -- please may I ask the same questions as above?

A

[mgorny]

We actually use prebuilt docs for CPython on Gentoo, so we're not affected — and in general, we don't really mind requiring newer versions for doc builds.

[hugovk]

Regarding Debian: we still have many packages which fail to build with Sphinx 7.x. See this list: bugs.debian.org/cgi-bin/pkgreport.cgi?tag=sphinx7.1;users=python-modules-team@lists.alioth.debian.org.

I want to give some more time for the maintainers to fix these bugs, but it's a few weeks, hopefully not months.

@mitya57 Of the stable Debian releases, which support any of Python 3.11+, and which Sphinx versions do they currently have?

Am I right it's only Debian 11 Bullseye with Python 3.11, and it has Sphinx 5.3.0? Thanks.

[mitya57]

No, Debian 11 Bullseye has Python 3.9 and Sphinx 3.4.

Debian 12 Bookworm has 3.11.2, but I don't know if there are plans to update it to a newer 3.11.x patch release — maybe @doko42 will tell better. This release has Sphinx 5.3 and it definitely won't be upgraded in stable series.

Debian unstable (Sid) and testing (Trixie) have Python 3.11.5 and Sphinx 5.3 at the moment, but as I mentioned above I will upgrade Sphinx eventually.

[hugovk]

Thanks all, so right now we're looking at something like this:

Sphinx for distro	CPython 3.11	CPython 3.12	CPython 3.13
Debian 12 Bookworm	5.3.0
Debian Sid (unstable)	5.3.0
Debian 13 Trixie (testing)	5.3.0
Fedora 37	5.0.2
Fedora 38	5.3.0
Fedora 39		6.2.1
Fedora 40		7.1.2
Gentoo	n/a	n/a	n/a
openSUSE Leap	4.2.0
openSUSE Tumbleweed	7.2.4	7.2.4
RHEL	n/a	n/a	n/a
Minimum Sphinx version	4.2.0	6.2.1

[vstinner]

#108184 introduces a nice improvement to https://docs.python.org/dev/library/string.html by linking the tokens.

I don't think that adding a few links in these tokens is worth the trouble. We should do our best to maximize support of old Sphinx versions.

You can add links under the table if it matters to you.

[mcepl]

In openSUSE Leap we've also Python 3.11 and Sphinx 4.2, so maybe that will require the inclusion of a new version of Sphinx there. But as far as I can tell, it won't be very painful.

This is not exactly correct: it is more or less true for openSUSE/Leap, not for SLE-15. Python 3.11 is an additional module, so the base Python is 3.6.15 (yes, I know, a bloody enterprise distro … and I am lucky with 3.6, I have to still support 3.4 in SLE-12), but fortunately we have managed to port Sphinx 4.2.0 to it.

And yes, as a distro maintainer, I am generally against this rush for the highest version. It brings a lot of grief to us for not that much benefits (you know, there is awfully lot you can write using even plain docutils ;), probably still too much).

[hugovk]

And yes, as a distro maintainer, I am generally against this rush for the highest version. It brings a lot of grief to us for not that much benefits (you know, there is awfully lot you can write using even plain docutils ;), probably still too much).

Yes, we're definitely mindful of not wanting to cause grief, which is why we're surveying you all to ask what distros are already using. And then being cautious by picking the lowest version, which would be 4.2.0, so that you don't need to do any work to upgrade to a newer Sphinx.

How does that sound?

[mcepl]

How does that sound?

Yes, of course, 4.2.0 is OK with us. I was just expressing general distaste for the rush to the highest versions of tools used.

[hugovk]

The minimum is now 4.2.0 for Python 3.11.

Thanks all for the help!