Stale header and footer for some documentation variants/versions

[amotl]

Hi there,

@msbt made me aware that some build variants of the documentation show up with outdated header and footer components. Thank you!

The header and footer should be rendered properly by using ESI.

When I take a closer look at https://readthedocs.org/projects/crate-jdbc/versions/, it seems like for those two variants, the header and footer seem to be fine:

• https://crate-jdbc.readthedocs.io/en/latest/
• https://crate-jdbc.readthedocs.io/en/1.14/

However, for those two variants, header and footer are stale:

• https://crate-jdbc.readthedocs.io/en/2.5/
• https://crate-jdbc.readthedocs.io/en/2.6/

With kind regards,
Andreas.

/cc @norosa

[amotl]

For https://crate-jdbc.readthedocs.io/en/2.5/, I just wiped the build environment on RTD and invoked a rebuild. However, this didn't change anything.

Maybe a stale Fastly cache is also involved here in any way?

/cc @WalBeh

[msbt]

I think Fastly could only be the cause if you checked the crate.io domain, but if it's directly on readthedocs.io, it must be something else.

[nomicode]

@amotl you need to make sure that the corresponding release branches are configuring sphinx correctly. older configurations may be pinning older versions of the theme. this is usually the cause. to fix this, you need to backport changes to the release branch to bring the sphinx config up-to-date

NOTE! sometimes this breaks stuff. bcus pushing to an older release branch will re-trigger the CI tests for that branch. in some cases, this causes problems because newer dependencies that get pulled in break the tests. then to get those tests to pass, we need to do a bunch of work on the CI or the code. this isn't always desirable, esp if the release is quite old and we're no longer really maintaining it

(if the release is old enough for us to not want to fix broken code tests, it is a good candidate for us removing as an active doc version imo - but you need to discuss this with the release manager for the project)

speak to @seut for more guidance. he's the person I've annoyed the most 😅 in the past by breaking PHP tests on old versions because I backported changes to the sphinx config

[amotl]

Hi Naomi,

this sounds complicated. Do you see an easy way to rebuild older documentation variants with the current infrastructure/theme?

Edit:

you need to make sure that the corresponding release branches are configuring sphinx correctly. older configurations may be pinning older versions of the theme. this is usually the cause. to fix this, you need to backport changes to the release branch to bring the sphinx config up-to-date

Ah all right. Will try this!

With kind regards,
Andreas.

[amotl]

However, for those two variants, header and footer are stale:

• https://crate-jdbc.readthedocs.io/en/2.5/
• https://crate-jdbc.readthedocs.io/en/2.6/

Wouldn't crate-docs-theme>=0.7 install the most recent version of the theme?

• https://github.com/crate/crate-jdbc/blob/2.5/docs/requirements.txt#L4
• https://github.com/crate/crate-jdbc/blob/2.6/docs/requirements.txt#L4

[nomicode]

to debug stuff like this, head to the RTD builds page:

https://readthedocs.org/projects/crate-jdbc/builds/

then, find the latest build corresponding to the version that is broken. for 2.6 (at the time of writing), that's:

https://readthedocs.org/projects/crate-jdbc/builds/13877032/

then, select view raw:

https://readthedocs.org/api/v2/build/13877032.txt

I am a little surprised to see the build downloading so many versions of the theme package:

[...]
Collecting crate-docs-theme>=0.7
  Downloading crate_docs_theme-0.15.1-py3-none-any.whl (998 kB)
  Downloading crate_docs_theme-0.15.0-py3-none-any.whl (4.1 MB)
  Downloading crate_docs_theme-0.14.1-py3-none-any.whl (4.0 MB)
Collecting sphinx-sitemap==2.2.0
  Downloading sphinx-sitemap-2.2.0.tar.gz (6.1 kB)
Collecting crate-docs-theme>=0.7
  Downloading crate_docs_theme-0.14.0-py3-none-any.whl (4.0 MB)
  Downloading crate_docs_theme-0.13.3-py3-none-any.whl (3.9 MB)
  Downloading crate_docs_theme-0.13.2-py3-none-any.whl (3.9 MB)
  Downloading crate_docs_theme-0.13.1-py3-none-any.whl (3.9 MB)
  Downloading crate_docs_theme-0.13.0-py3-none-any.whl (3.9 MB)
  Downloading crate_docs_theme-0.12.0-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.11.0-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.17-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.16-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.15-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.14-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.13-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.12-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.11-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.10-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.9-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.8-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.7-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.6-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.5-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.4-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.3-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.2-py3-none-any.whl (3.8 MB)
  Downloading crate_docs_theme-0.10.1-py3-none-any.whl (3.8 MB)
Collecting sphinxcontrib-plantuml==0.14
  Downloading sphinxcontrib-plantuml-0.14.tar.gz (7.3 kB)
Collecting crate-docs-theme>=0.7
  Downloading crate_docs_theme-0.10.0-py3-none-any.whl (3.8 MB)
[...]

shortly after that, you can see which version it settled on:

Successfully installed PyYAML-5.4.1 argh-0.26.2 crate-docs-theme-0.10.0 livereload-2.6.3 pathtools-0.1.2 port-for-0.3.1 setuptools-40.9.0 sphinx-1.7.4 sphinx-autobuild-0.6.0 sphinx-sitemap-0.3.1 sphinxcontrib-plantuml-0.14 tornado-6.1 watchdog-2.1.2

so, 0.10.0

there must be some interactions between dependencies I don't understand going on here which causes Pip to satisfy the requirement with this version

a good first step at debugging issues like this is to reset the build environment manually. go here:

https://readthedocs.org/projects/crate-jdbc/versions/

select Edit next to 2.6

select wipe on the subsequent screen. then confirm the operation

now, go back to the builds tab:

https://readthedocs.org/projects/crate-jdbc/builds/

select "2.6" from the drop-down on the right, and then select Build Version

you'll get taken to a new build page. for me, it was:

https://readthedocs.org/projects/crate-jdbc/builds/13896577/

wait for this to complete, then select view raw again:

https://readthedocs.org/api/v2/build/13896577.txt

I get the same result: version 0.10.0

compare this to the raw build logs for the "latest" docs:

https://readthedocs.org/api/v2/build/13877030.txt

this seems like a clue:

INFO: pip is looking at multiple versions of sphinxcontrib-plantuml to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of sphinx-sitemap to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of sphinx-copybutton to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of crate-docs-theme to determine which version is compatible with other requirements. This could take a while.

end result:

Successfully installed PyYAML-5.4.1 argh-0.26.2 crate-docs-theme-0.14.1 livereload-2.6.3 pathtools-0.1.2 port-for-0.3.1 setuptools-40.9.0 sphinx-autobuild-0.6.0 sphinx-sitemap-2.2.0 sphinxcontrib-plantuml-0.19 tornado-6.1 watchdog-2.1.2

differences:

• "2.6"
  • crate-docs-theme-0.10.0
  • sphinx-1.7.4
  • sphinx-sitemap-0.3.1
  • sphinxcontrib-plantuml-0.14
• "latest"
  • crate-docs-theme-0.14.1
  • sphinx-sitemap-2.2.0
  • sphinxcontrib-plantuml-0.19

the thing that stands out to me: the two builds are using different versions of Sphinx. the fact that the "latest" build doesn't mention installing Sphinx indicates to me that the version of Sphinx the build is using comes pre-installed in the environment by RTD. conversely, the "2.6" build is forcing a particular Sphinx version

compare the requirements.txt files for both versions:

• https://github.com/crate/crate-jdbc/blob/2.6/docs/requirements.txt
• https://github.com/crate/crate-jdbc/blob/master/docs/requirements.txt

the only difference is the version of Sphinx that has been pinned:

• "2.6" has sphinx==1.7.4
• "latest" has sphinx==1.8.5

that change was made here:

36d968a

my proposed solution is to cherry-pick this commit to both older release lines. I staged some PRs to do this:

• backport for 2.6
• backport for 2.5

if the Travis checks pass, then we're good to go. once they're merged and the docs get rebuilt, I believe this issue will be resolved. if they don't pass, then there is potentially some breakage with the driver dependencies. and we'll have to discuss what to do

---

we could probably skip a bunch of these steps in the future if we modified the theme so that it wrote an HTML comment to every page saying what version of the theme it was used

would be a quick way to check if the theme is bust or the build is bust

[nomicode]

1.14 seems to have been unscathed because it is old enough to have never received a change that pins Sphinx to a specific version. see https://github.com/crate/crate-jdbc/blob/1.14/docs/requirements.txt

[nomicode]

this line in the theme setup.py is what causes Pip to use an older version of the theme:

https://github.com/crate/crate-docs-theme/blob/5302edf75046895d0cb8c49439ed2e00f8eaf725/setup.py#L60

basically, Pip sees sphinx==1.7.4 in docs/requirements.txt and works backwards through the different versions of the crate-docs-theme package until it can satisfy both sets of rules

[amotl]

Will #344/#345 fix this?

[nomicode]

@amotl I manually kicked off the builds after merging my backports. I inspected the raw build output and both now use crate-docs-theme-0.14.1. so I think this is fixed. will leave for you to verify and close

I created an off-shoot issue: crate/crate-docs-theme#294

[amotl]

Hi Naomi,

I believe everything is much better now. While the footer on https://crate-jdbc.readthedocs.io/en/1.14/ still looks slightly different (line-height), I believe it is fine to close this issue because, apparently, all outdated versions I referenced in my OP now at least have content-wise updated footers.

Thank you!

With kind regards,
Andreas.

P.S.: If you still see anything to be done here, feel free to reopen, @msbt.