Switch to the latest sphinx

[wRAR]

The docs fail to build with the current Sphinx (7.2.6):

reading sources... [ 48%] topics/downloader-middleware
Extension error (scrapydocs):
Handler <function collect_scrapy_settings_refs at 0x7f81fc663a60> for event 'doctree-read' threw an exception (exception: Next node is not a target)

So we should update deps in docs/requirements.txt, fix this (and maybe others) problem and make sure the docs are built correctly.

[wRAR]

The exception is caused by some change in Sphinx 7.2.0 that leads to a different order of nodes when there are two .. setting directives in a row (index,target,index,target in older versions but index,index,target,target in newer ones) which breaks our code. No idea yet why this happens and if it's expected.

[wRAR]

Bisected to sphinx-doc/sphinx#10478

[wRAR]

sphinx-doc/sphinx#11797

[wRAR]

Per the comment in the upstream issue we need to change the logic in get_setting_target() and it should be relatively easy to do that.

As an aside, I'm not sure if the HTML that is currently generated for multiple setting directives in a row is correct, see e.g. https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#httpproxymiddleware-settings and where are the #std-setting-HTTPPROXY_ENABLED and #std-setting-HTTPPROXY_AUTH_ENCODING elements.

[Gallaecio]

As an aside, I'm not sure if the HTML that is currently generated for multiple setting directives in a row is correct, see e.g. https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#httpproxymiddleware-settings and where are the #std-setting-HTTPPROXY_ENABLED and #std-setting-HTTPPROXY_AUTH_ENCODING elements.

As far as I know, <span id="…"></span> is a valid approach to set an anchor anywhere on an HTML document.

[wRAR]

@Gallaecio I mean those two elements even have different depth, and #std-setting-HTTPPROXY_AUTH_ENCODING is inside #httpproxy-enabled, so this HTML looks like something that works approximately (because the links are near the place we want) but is not strictly what we want.

[Rotzbua]

@wRAR With Sphinx v7 a lot is being reworked. This leads to many side effects. I would advise you to migrate to the latest v6.

[wRAR]

@Rotzbua do you suggest migrating to 7.x later?

[Gallaecio]

Fixed by #6200