Require Sphinx 6.2 to build Python 3.13 documentation

[vstinner]

I propose to require Sphinx 6.2.0 to build Python 3.13 documentation.

PEP 594 scheduled the removal of the imghdr module in Python 3.13. Problem: Sphinx EPUB builder used it until Sphinx 6.2.

Sphinx 6.2 got fixed: it no longer uses this deprecated module:

• imghdr is deprecated and slated for removal in Python 3.13 sphinx-doc/sphinx#10440 (comment)
• sphinx-doc/sphinx@a502e75
• https://github.com/sphinx-doc/sphinx/releases/tag/v6.2.0

This change is ok for Fedora and RHEL will follow Fedora, and it's ok for OpenSuSE.

In May 2022, Debian Stable used Sphinx 3.4 and Ubuntu LTS used Sphinx 3.5.

In May 2022, Python was modified to require Sphinx 3.2 or newer: see issue #86986.

See also details in the imghdr module removal PR.

---

Alternatives:

• Run Sphinx with Python older than 3.13 (ex: Python 3.12) which still had the imghdr module
• Don't remove the imghdr module in Python 3.13
• Provide the imghdr module on PyPI, as it was done with nntplib (on PyPI) -- old Sphinx versions would still be supported in this case
• For Linux distributions: backport the Sphinx 6.2 change to older Sphinx version to support Python 3.13 without imghdr.

Linked PRs

• gh-104818: Require Sphinx 6.2 to build the doc #104819

[hugovk]

I think this is fine: it can be only for 3.13, not due out until October 2024, and Sphinx 6.2 will be 18 months old by then.

[vstinner]

I propose to require Sphinx 6.2.0 to build Python 3.13 documentation.

Sphinx 6.2.0 was released at Apr 23, 2023. Today, it's only 1 month old. But when Python 3.13 final will be released (October 2024), it will be 1 year and 6 months old.

[vstinner]

cc @JulienPalard @deronnax @doko42 @encukou

[vstinner]

I think this is fine: it can be only for 3.13, not due out until October 2024, and Sphinx 6.2 will be 18 months old by then.

Would you mind to officially approve the PR #104819 in that case?

[gvanrossum]

We don't normally build the docs with the Python built from the current repo do we? The Makefile seems to say PYTHON = python3 and that would normally be the installed Python (presumably the same one used for the various generating scripts). So this doesn't seem to be urgent. Then why hurry?

[hugovk]

No, we don't build the docs with Python built from the current repo, but we do run doctest on the CI which needs that Python to actually test that Python.

For example, see the imghdr removal PR (#104777) that fails with old Sphinx:

PATH=./venv/bin:$PATH sphinx-build -b doctest -d build/doctrees  -j auto  -W --keep-going . build/doctest 
Running Sphinx v4.5.0

Extension error:
Could not import extension sphinx.builders.epub3 (exception: No module named 'imghdr')
make[1]: *** [Makefile:50: build] Error 2
make[1]: Leaving directory '/home/runner/work/cpython/cpython/Doc'
Testing of doctests in the sources finished, look at the results in build/doctest/output.txt
make: *** [Makefile:[12](https://github.com/python/cpython/actions/runs/5082380617/jobs/9132034271?pr=104777#step:9:13)8: doctest] Error 1
make: Leaving directory '/home/runner/work/cpython/cpython/Doc'
Error: Process completed with exit code 2.

https://github.com/python/cpython/actions/runs/5082380617/jobs/9132034271?pr=104777

[vstinner]

According to packages.debian.org, Debian provides:

• stable: python3-sphinx (3.4.3-2) and python3.9; python3 = 3.9.2-3
• testing: python3-sphinx (5.3.0-4) and python3.11; python3 = 3.11.2-1+b1
• unstable: python3-sphinx (5.3.0-4) and python3.12; python3 = 3.11.2-1+b1
• experimental: python3-sphinx (7.0.1-1) and python3.12

I see that Sphinx is slowly being upgraded from 3.4 to 5.3, and then later a migration to 7.0 may happen. While Sphinx is upgrade to 5.3 in testing, Python is upgraded from 3.9 to 3.11.

I imagine that in the future, a similar move will happen with the migration from Sphinx 5.3 to 6.2 (or newer) and Python from 3.11 to 3.13 (or maybe first to 3.12).

Again, if Debian is stuck with an older Sphinx version, it's easy to backport the change which removed the dependency to the imghdr dependency. Or someone can provide imghdr on PyPI, Debian can create a new dedicated package for it, and then add a build dependency on it (it's not needed to run Python, obviously).

Note: a quick & dirty hack is to just copy the Lib/imghdr.py file from Python 3.12 to the right place in Sphinx. But it should not land in production, right? ;-)

[vstinner]

I looked again at the whole problem. I understood that Sphinx 6.2 was stricky required to build the Python documentation on Python 3.13. I was wrong. It's only required for the specific GHA Doctest CI job: only this job runs with Python 3.13. In fact, the GHA Doc job which only builds the documentation runs with Sphinx 3.2 which remains compatible with Python 3.11.

Since upgrading Sphinx is not strictly required to remove the imghdr module, I close this issue.

Obviously, if someone has another good rationale to upgrade Sphinx, please open a new separated issue with that rationale. But again, the imghdr removal is not a good rationale :-)

[hugovk]

Ah yes, that's a good solution in PR #104777 to unblock imghdr removal: Sphinx 6.2 in requirements.txt, used for building the docs on the CI and for doctest.

And maintain support for 3.2 by keeping needs_sphinx = '3.2' in Doc/conf.py and 3.2 in Doc/requirements-oldest-sphinx.txt to make sure they still build.

We can consider bumping the minimum Sphinx more incrementally.

For example, can we already move to Sphinx 5.3 for main/3.13, 3.12 and 3.11?

[CAM-Gerlach]

I'd think that would think 5.3 as the minimum for 3.11, and maybe 3.12 too would be too aggressive, as it was released a few days before the final release of Python 3.11.0 itself, and it is an already released feature version. For 3.12 we could perhaps bump it to 4.5.0 though, and maybe go 5.3.0 with 3.13.

On the other hand, the normal requirements.txt Sphinx version for the web docs is still 4.5.0 for 3.12 and 3.11. The one of the main benefits of also testing the minimum version, IMO, is it gives us more freedom to upgrade that and keep all the (bug-fix) maintained the build branches in sync on the same Sphinx version, preferably the latest. So IMO we should backport that version bump to 6.2.0, as well as future ones when desired. just like with other docs changes.

Also, re-closing the issue for now with the correct close state.