Pin Jinja2 to fix docs build

[hugovk]

BPO	47138
Nosy	@ned-deily, @ambv, @zooba, @hugovk, @miss-islington, @m-aciek
PRs	[3.7] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32111 [3.8] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32109 [3.9] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32153 [3.10] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32154 bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms #32161 [3.10] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32163 [3.8] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32167 [3.7] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32168 [3.9] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32169 [3.7] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms #32182 [3.8] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32183

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-03-28.08:17:25.467>
labels = ['3.7', '3.8', '3.9', '3.10', 'docs']
title = 'Pin Jinja2 to fix docs build'
updated_at = <Date 2022-04-08.13:06:24.310>
user = 'https://github.com/hugovk'

bugs.python.org fields:

activity = <Date 2022-04-08.13:06:24.310>
actor = 'lukasz.langa'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2022-03-28.08:17:25.467>
creator = 'hugovk'
dependencies = []
files = []
hgrepos = []
issue_num = 47138
keywords = ['patch']
message_count = 14.0
messages = ['416149', '416151', '416158', '416167', '416171', '416194', '416195', '416196', '416197', '416209', '416233', '416235', '416285', '416983']
nosy_count = 7.0
nosy_names = ['ned.deily', 'docs@python', 'lukasz.langa', 'steve.dower', 'hugovk', 'miss-islington', 'Maciej Olko']
pr_nums = ['32111', '32109', '32153', '32154', '32161', '32163', '32167', '32168', '32169', '32182', '32183']
priority = 'normal'
resolution = 'fixed'
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue47138'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[hugovk]

The docs build on GitHub Actions is passing on main but has started failing for the 3.7 - 3.10 branches:

Missing the required blurb or sphinx-build tools.
Please run 'make venv' to install local copies.

make[1]: *** [Makefile:50: build] Error 1

https://github.com/python/cpython/actions/workflows/doc.yml
https://github.com/python/cpython/runs/5714593700?check_suite_focus=true

This is because the PATH=./venv/bin:$PATH sphinx-build --version check is failing:

/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/util/docutils.py:45: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  __version_info__ = tuple(LooseVersion(docutils.__version__).version)
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/Doc/./venv/bin/sphinx-build", line 5, in <module>
    from sphinx.cmd.build import main
  File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/cmd/build.py", line 25, in <module>
    from sphinx.application import Sphinx
  File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/application.py", line 42, in <module>
    from sphinx.registry import SphinxComponentRegistry
  File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/registry.py", line 24, in <module>
    from sphinx.builders import Builder
  File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 26, in <module>
    from sphinx.util import import_object, logging, rst, progress_message, status_iterator
  File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/util/rst.py", line 22, in <module>
    from jinja2 import environmentfilter
ImportError: cannot import name 'environmentfilter' from 'jinja2' (/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/jinja2/__init__.py)

This is because 3.10 and 3.9 are still using old versions of Sphinx (3.2.1 and 2.4.4 respectively) which is incompatible with the newest Jinja2:

pallets/jinja#1630

main is using a newer Sphinx (4.2.0), so it passes.

Bumping to Sphinx 4.5.0 will fix both 3.9 and 3.10. 3.7 and 3.8 are affected too. We can bump main to use 4.5.0 as well.

[hugovk]

re: https://bugs.python.org/issue45618#msg416017

Maciej Olko, please could you update these PRs to keep the Sphinx version the same and only pin Jinja2 to match?

3.7: #32109
3.8: #32111

I guess update them to use this bpo too. Thanks!

[m-aciek]

I've update both PRs to contain the pin only.

[ned-deily]

Thank you both for addressing this. The main reason for minimizing changes to the docs build chain for brances in bugfix or security-fix-only phases is that many distributors of Python also release the Python docs as a package and they do not necessarily rely on using our fairly-recent venv step in the Docs Makefile. In particular, they want to use the versions of Sphinx et al that they release as packages. If we change the required versions for these mid-stream, they not only have to update the docs themselves but have to go through the process of updating the other packages (like Sphinx) which may have other consequences. This can also impact our own on-line doc building process. (Version updates for a new Python feature release prior to its entering beta phase are not a problem.) So it is best to do the minimal amount of version updating for releases in the field, preferably none (as in pinning existing versions). In that spirit, we should also just pin the Jinja2 version for 3.10 and 3.9.

[hugovk]

I cherry picked Maciej's 3.8 commit into 3.9 and 3.10 PRs:

3.9: #32153
3.10: #32154

And marking this issue as not relevant to 3.11; instead I bumped the Sphinx version in bpo-47126 / #76305 as it's directly relevant there.

Thanks!

[ned-deily]

New changeset 25f00bf by m-aciek in branch '3.7':
bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32111)
25f00bf

[ned-deily]

New changeset 9194a7b by Hugo van Kemenade in branch '3.10':
bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32154)
9194a7b

[ned-deily]

New changeset 25c40bd by Hugo van Kemenade in branch '3.9':
bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32153)
25c40bd

[ned-deily]

Thanks, everyone! Merged for 3.10.5, 3.9.13, and 3.7.14. I'll leave the issue in pending state until merged for 3.8.14.

[zooba]

New changeset 295114d by Steve Dower in branch 'main':
bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161)
295114d

[zooba]

New changeset 0dfabf9 by Steve Dower in branch '3.10':
bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161)
0dfabf9

[miss-islington]

New changeset d2c6a44 by Miss Islington (bot) in branch '3.9':
bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161)
d2c6a44

[ned-deily]

New changeset d97497b by Steve Dower in branch '3.7':
bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32182)
d97497b

[ambv]

New changeset d35af52 by m-aciek in branch '3.8':
[3.8] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32109)
d35af52

[kumaraditya303]

@hugovk Can this be closed now ?

[hugovk]

We're still waiting for #32183 to be merged for 3.8.

Ping 3.8 RM @ambv.

[hugovk]

#32183 is now merged! Closing, thanks all!