Provide a version drop-down menu in documentation #23

billsacks · 2018-12-09T13:12:15Z

This uses a new documentation theme together with some javascript to provide a version drop-down menu in the documentation build.

This is based on what @mnlevy1981 did for MARBL in marbl-ecosys/MARBL@ddd346f . That, in turn, was based on what's done in https://github.com/unidata/metpy.

Note that the necessary version of sphinx-rtd-theme is only available on a branch right now (in an open PR to that project). Rather than requiring installation of this, I have pulled it in as a subtree:

git subtree add --prefix doc/source/_themes/sphinx_rtd_theme git@github.com:rtfd/sphinx_rtd_theme.git version-dropdown --squash

This relies on having a versions.json file in the documentation build (in path, cism-in-cesm/versions/versions.json). Its current contents are:

{
    "master": "master",
    "release-cesm2.0": "CESM2.0"
}

…46bba5 git-subtree-dir: doc/source/_themes/sphinx_rtd_theme git-subtree-split: b46bba51649fd15b90e6938b57a1e96f7063c81f

…e/_themes/sphinx_rtd_theme'

Otherwise we get warnings like this: WARNING: autodoc: failed to import module 'test' from module 'test_py_module'; the following exception was raised: No module named 'test_py_module' /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/demo_docs/source/demo.rst:261: WARNING: Footnote [4] is not referenced. /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/demo_docs/source/demo.rst:267: WARNING: Footnote [11] is not referenced. /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/demo_docs/source/demo.rst:18: WARNING: Undefined substitution referenced: "problematic". /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/demo_docs/source/demo.rst:261: WARNING: Unknown target name: "5". /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/demo_docs/source/demo.rst:304: WARNING: Unknown target name: "body elements". /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/demo_docs/source/demo.rst:316: WARNING: Unknown target name: "hyperlink reference without a target". looking for now-outdated files... none found pickling environment... done checking consistency... /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/README.rst: WARNING: document isn't included in any toctree /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/demo_docs/source/index.rst: WARNING: document isn't included in any toctree /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/tests/roots/test-basic/index.rst: WARNING: document isn't included in any toctree /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/tests/roots/test-empty/index.rst: WARNING: document isn't included in any toctree /Users/sacks/cism-code/cism-wrapper/cism-wrapper_master/doc/source/_themes/sphinx_rtd_theme/tests/roots/test-missing-toctree/index.rst: WARNING: document isn't included in any toctree See also readthedocs/sphinx_rtd_theme#142

The files in _static and _templates were pulled in directly from marbl-ecosys/MARBL@ddd346f, without any changes. The changes in conf.py were based on the changes in that MARBL commit. A few notes about what seems to be needed: - It seems that the version and/or release (probably version) listed at the top of the file needs to match the string used in html_theme_options['versions'] - It seems that we really do need to list every version in html_theme_options['versions'] in order for the drop-down menus to be created correctly - This seemed to work regardless of whether I had a versions.json file in the build directory Update 2018-12-08: The above notes are no longer true now that I have the javascript working correctly (with correct paths for our documentation build), as I fixed in a later commit on this branch.

- No longer have an -i argument (now just specify that via extra path elements in the -r argument) - Now have a versions subdirectory

A key difference driving some of these changes is that, with our directory structure, there is an extra 'html' subdirectory in the paths. Specific changes are: - In conf.py: Add '_static' to html_static_path. This is needed in order to copy the javascript into the build - In conf.py: Set the initial values of 'versions' in a way that can stay the same for all branches. Note that these initial values aren't really important, because they will be replaced by the javascript. - In footer.html: Change the path to be correct given this extra 'html' subdirectory in the paths - In pop_ver.js: Get rid of the startsWith/substring logic that wasn't doing anything (as far as I can tell). Instead, leverage a "smarter" versions.json file that provides any desired renames. See below for the file I'm currently using. - In pop_ver.js: Fix the URLs to be correct for our directory structure, which has 'html' in the paths. - In pop_ver.js: Fix the URL for the current version. This doesn't seem to be important in most cases: since the current version starts out being the selected version, clicking on it doesn't do anything. However, the old, incorrect link caused problems in Firefox when I navigated to a page in a certain version, then clicked on a different version, then hit back (at this point, the version in the menu was the different version), then clicked on the current version: this gave a page not found error. - In pop_ver.js: Add a bunch of comments I currently have the following versions.json file in the 'versions' subdirectory of the documentation build; note that this now maps a directory name onto a version name: { "master": "master", "release-cesm2.0": "CESM2.0", "release-cesm2.1": "CESM2.1" } (Note that the separate CESM2.0 vs. CESM2.1 was just done for the purposes of testing to make sure we could add a third version that isn't explicitly listed in conf.py: in reality we still just have a single release version of the documentation.)

billsacks · 2018-12-09T13:39:30Z

I tried pulling sphinx_rtd_theme in via manage_externals, but ran into problems related to the fact that the .gitattributes file in that repository results in a conversion of line endings in docs/make.bat on my Mac - so immediately upon cloning, this file is modified in my working tree, with git giving the message:

warning: CRLF will be replaced by LF in docs/make.bat.
The file will have its original line endings in your working directory.

Then when manage_externals tries to checkout a different hash, it can't:

    error: Your local changes to the following files would be overwritten by checkout:
    	docs/make.bat
    Please commit your changes or stash them before you switch branches.

So I'll stick with the subtree for now.

This changes the documentation theme to use the readthedocs theme, with some JavaScript that provides capabilities for a dropdown menu allowing you to select between different versions. This mimics the changes in ESCOMP/CISM-wrapper#23

This changes the documentation theme to use the readthedocs theme, with some JavaScript that provides capabilities for a dropdown menu allowing you to select between different versions. This mimics the changes in ESCOMP/CISM-wrapper#23 The changes in this commit assume that sphinx_rtd_theme is installed in _themes. However, we are not going to do this here: instead, we'll rely on a pip install of an appropriate branch of sphinx_rtd_theme, using: pip install git+https://github.com/esmci/sphinx_rtd_theme.git@version-dropdown-with-fixes A follow-up commit will fix that.

@mvertens

Documentation: provide a version dropdown menu and change theme The documentation of cime is starting to diverge on the various important branches (master, maint-5.6, and ufs_release_v1.0). This PR, along with similar PRs for maint-5.6 and ufs_release_v1.0, and some changes to the gh-pages branch, support having versioned documentation, with a dropdown menu. This also changes the documentation theme to a theme that supports this version dropdown. Note that this requires a branch of the sphinx_rtd_theme; I will update the cime wiki on building the documentation to note the new pip install step needed for that. (See below for more details.) At a high level, the way the versioned documentation works is to have separate subdirectories in the gh-pages branch for each version of the documentation we want to support. There is then a bit of JavaScript code which uses a json file in the gh-pages branch to determine which versions exist and how these should be named in the dropdown menu. This changes the documentation build process slightly; I will update the wiki to reflect these changes. @mvertens and I made the various changes that you can see in `doc/source` in this PR, which mainly borrowed from ESCOMP/CISM-wrapper#23. That, in turn, is a slight modification of an implementation provided by @MNLevy for the MARBL documentation, which in turn borrowed from an implementation put together by Unidata (credit where credit is due). As mentioned above, I am not aware of out-of-the-box support for a version pull-down in out-of-the-box sphinx themes (though the last time I looked was in Fall, 2018, so there may be something available now). However, support for a version dropdown exists in an open PR in the sphinx readthedocs theme repository: readthedocs/sphinx_rtd_theme#438. I have pushed this branch to a new repository in ESMCI (https://github.com/ESMCI/sphinx_rtd_theme) to facilitate long-term maintenance of this branch in case it disappears from the official sphinx_rtd_theme repository. I have also cherry-picked a commit onto that branch, which is needed to fix search functionality in sphinx1.8 (from readthedocs/sphinx_rtd_theme#672) (which is another reason for maintaining our own copy of this branch). The branch in this repository is now named version-dropdown-with-fixes (branching off of the version-dropdown branch in the sphinx_rtd_theme repository). I will update the wiki to give the additional pip install command needed to install this repository before doing the build. In the long-term, I am a little concerned about using this theme that isn't showing any signs of being merged to the main branch of the readthedocs theme, but this has been working for us in other projects for the last 2 years, so I feel this is a reasonable approach in the short-medium term. A few changes were also needed in the gh-pages branch: - A `versions` directory was introduced at the top level, with subdirectories for each documentation version we want to support. We are using the convention that these subdirectories should have names that match the corresponding cime branches, to avoid uncertainty. - A top-level index.html page was introduced that redirects to the master version of the documentation: ```html <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="refresh" content="0; url=versions/master/html/index.html" /> </head> </html> ``` - Other than those files and an empty `.nojekyll` file, everything else was removed from the top level directory of the documentation. - In the versions directory, we also introduced a `versions.json` file. This lists the versions and provides a mapping between directory names and the names used in the pull-down menu: ```json { "master": "master", "maint-5.6": "cime5.6", "ufs_release_v1.0": "ufs_release_v1.0" } ``` - The complete contents of each version subdirectory are generated by the documentation build script. Thus, it is safe to remove the contents of (e.g.) `versions/master/` before rebuilding the documentation for that version. The documentation build is done similarly to before, but now providing a BUILDDIR argument to the make command in order to build in the correct subdirectory. For example, when building the documentation for master, you would provide `BUILDDIR=/path/to/cime_gh_pages_repo/versions/master/`. Test suite: none Test baseline: n/a Test namelist changes: none Test status: bit for bit Fixes none User interface changes?: none Update gh-pages html (Y/N)?: Y (already done) Code review:

This changes the documentation theme to use the readthedocs theme, with some JavaScript that provides capabilities for a dropdown menu allowing you to select between different versions. This mimics the changes in ESCOMP/CISM-wrapper#23 The changes in this commit assume that sphinx_rtd_theme is installed in _themes. However, we are not going to do this here: instead, we'll rely on a pip install of an appropriate branch of sphinx_rtd_theme, using: pip install git+https://github.com/esmci/sphinx_rtd_theme.git@version-dropdown-with-fixes A follow-up commit will fix that.

This changes the documentation theme to use the readthedocs theme, with some JavaScript that provides capabilities for a dropdown menu allowing you to select between different versions. This mimics the changes in ESMCI/cime#3439, which in turn was based on ESCOMP/CISM-wrapper#23.

@mnlevy1981

Bring documentation source to master 1. Bring documentation source to master: Pulls in the source from https://github.com/escomp/ctsm-docs. This is important so that documentation can remain in sync with changes in the model code. Images are stored here using git-lfs (Git Large File Storage). I also made some minor fixes to get the pdf build of the tech note working. 2. Use a different documentation theme that supports a version dropdown menu, and add the code needed to support this versioning on the documentation web pages. At a high level, the way the versioned documentation works is to have separate subdirectories in the gh-pages branch of the ctsm-docs repository for each version of the documentation we want to support. There is then a bit of JavaScript code which uses a json file in the gh-pages branch to determine which versions exist and how these should be named in the dropdown menu. Most of these changes were borrowed from ESMCI/cime#3439, which in turn borrowed from ESCOMP/CISM-wrapper#23, which in turn was a slight modification of an implementation provided by @mnlevy1981 for the MARBL documentation, which in turn borrowed from an implementation put together by Unidata (credit where credit is due). I am not aware of out-of-the-box support for a version pull-down in out-of-the-box sphinx themes (though the last time I looked was in Fall, 2018, so there may be something available now). However, support for a version dropdown exists in an open PR in the sphinx readthedocs theme repository: readthedocs/sphinx_rtd_theme#438. I have pushed this branch to a new repository in ESMCI (https://github.com/ESMCI/sphinx_rtd_theme) to facilitate long-term maintenance of this branch in case it disappears from the official sphinx_rtd_theme repository. I have also cherry-picked a commit onto that branch, which is needed to fix search functionality in sphinx1.8 (from readthedocs/sphinx_rtd_theme#672) (which is another reason for maintaining our own copy of this branch). The branch in this repository is now named version-dropdown-with-fixes (branching off of the version-dropdown branch in the sphinx_rtd_theme repository). In the long-term, I am a little concerned about using this theme that isn't showing any signs of being merged to the main branch of the readthedocs theme, but this has been working for us in other projects for the last 2 years, so I feel this is a reasonable approach in the short-medium term. The new process for building the documentation is given here: https://github.com/ESCOMP/CTSM/wiki/Directions-for-editing-CLM-documentation-on-github-and-sphinx Resolves #239

This changes the documentation theme to use the readthedocs theme, with some JavaScript that provides capabilities for a dropdown menu allowing you to select between different versions. This mimics the changes in ESMCI/cime#3439, which in turn was based on ESCOMP/CISM-wrapper#23.

@mnlevy1981

Bring documentation source to master 1. Bring documentation source to master: Pulls in the source from https://github.com/escomp/ctsm-docs. This is important so that documentation can remain in sync with changes in the model code. Images are stored here using git-lfs (Git Large File Storage). I also made some minor fixes to get the pdf build of the tech note working. 2. Use a different documentation theme that supports a version dropdown menu, and add the code needed to support this versioning on the documentation web pages. At a high level, the way the versioned documentation works is to have separate subdirectories in the gh-pages branch of the ctsm-docs repository for each version of the documentation we want to support. There is then a bit of JavaScript code which uses a json file in the gh-pages branch to determine which versions exist and how these should be named in the dropdown menu. Most of these changes were borrowed from ESMCI/cime#3439, which in turn borrowed from ESCOMP/CISM-wrapper#23, which in turn was a slight modification of an implementation provided by @mnlevy1981 for the MARBL documentation, which in turn borrowed from an implementation put together by Unidata (credit where credit is due). I am not aware of out-of-the-box support for a version pull-down in out-of-the-box sphinx themes (though the last time I looked was in Fall, 2018, so there may be something available now). However, support for a version dropdown exists in an open PR in the sphinx readthedocs theme repository: readthedocs/sphinx_rtd_theme#438. I have pushed this branch to a new repository in ESMCI (https://github.com/ESMCI/sphinx_rtd_theme) to facilitate long-term maintenance of this branch in case it disappears from the official sphinx_rtd_theme repository. I have also cherry-picked a commit onto that branch, which is needed to fix search functionality in sphinx1.8 (from readthedocs/sphinx_rtd_theme#672) (which is another reason for maintaining our own copy of this branch). The branch in this repository is now named version-dropdown-with-fixes (branching off of the version-dropdown branch in the sphinx_rtd_theme repository). In the long-term, I am a little concerned about using this theme that isn't showing any signs of being merged to the main branch of the readthedocs theme, but this has been working for us in other projects for the last 2 years, so I feel this is a reasonable approach in the short-medium term. The new process for building the documentation is given here: https://github.com/ESCOMP/CTSM/wiki/Directions-for-editing-CLM-documentation-on-github-and-sphinx Resolves ESCOMP#239

@mnlevy1981

Bring documentation source to master 1. Bring documentation source to master: Pulls in the source from https://github.com/escomp/ctsm-docs. This is important so that documentation can remain in sync with changes in the model code. Images are stored here using git-lfs (Git Large File Storage). I also made some minor fixes to get the pdf build of the tech note working. 2. Use a different documentation theme that supports a version dropdown menu, and add the code needed to support this versioning on the documentation web pages. At a high level, the way the versioned documentation works is to have separate subdirectories in the gh-pages branch of the ctsm-docs repository for each version of the documentation we want to support. There is then a bit of JavaScript code which uses a json file in the gh-pages branch to determine which versions exist and how these should be named in the dropdown menu. Most of these changes were borrowed from ESMCI/cime#3439, which in turn borrowed from ESCOMP/CISM-wrapper#23, which in turn was a slight modification of an implementation provided by @mnlevy1981 for the MARBL documentation, which in turn borrowed from an implementation put together by Unidata (credit where credit is due). I am not aware of out-of-the-box support for a version pull-down in out-of-the-box sphinx themes (though the last time I looked was in Fall, 2018, so there may be something available now). However, support for a version dropdown exists in an open PR in the sphinx readthedocs theme repository: readthedocs/sphinx_rtd_theme#438. I have pushed this branch to a new repository in ESMCI (https://github.com/ESMCI/sphinx_rtd_theme) to facilitate long-term maintenance of this branch in case it disappears from the official sphinx_rtd_theme repository. I have also cherry-picked a commit onto that branch, which is needed to fix search functionality in sphinx1.8 (from readthedocs/sphinx_rtd_theme#672) (which is another reason for maintaining our own copy of this branch). The branch in this repository is now named version-dropdown-with-fixes (branching off of the version-dropdown branch in the sphinx_rtd_theme repository). In the long-term, I am a little concerned about using this theme that isn't showing any signs of being merged to the main branch of the readthedocs theme, but this has been working for us in other projects for the last 2 years, so I feel this is a reasonable approach in the short-medium term. The new process for building the documentation is given here: https://github.com/ESCOMP/CTSM/wiki/Directions-for-editing-CLM-documentation-on-github-and-sphinx Resolves ESCOMP#239

@mnlevy1981

Bring documentation source to master 1. Bring documentation source to master: Pulls in the source from https://github.com/escomp/ctsm-docs. This is important so that documentation can remain in sync with changes in the model code. Images are stored here using git-lfs (Git Large File Storage). I also made some minor fixes to get the pdf build of the tech note working. 2. Use a different documentation theme that supports a version dropdown menu, and add the code needed to support this versioning on the documentation web pages. At a high level, the way the versioned documentation works is to have separate subdirectories in the gh-pages branch of the ctsm-docs repository for each version of the documentation we want to support. There is then a bit of JavaScript code which uses a json file in the gh-pages branch to determine which versions exist and how these should be named in the dropdown menu. Most of these changes were borrowed from ESMCI/cime#3439, which in turn borrowed from ESCOMP/CISM-wrapper#23, which in turn was a slight modification of an implementation provided by @mnlevy1981 for the MARBL documentation, which in turn borrowed from an implementation put together by Unidata (credit where credit is due). I am not aware of out-of-the-box support for a version pull-down in out-of-the-box sphinx themes (though the last time I looked was in Fall, 2018, so there may be something available now). However, support for a version dropdown exists in an open PR in the sphinx readthedocs theme repository: readthedocs/sphinx_rtd_theme#438. I have pushed this branch to a new repository in ESMCI (https://github.com/ESMCI/sphinx_rtd_theme) to facilitate long-term maintenance of this branch in case it disappears from the official sphinx_rtd_theme repository. I have also cherry-picked a commit onto that branch, which is needed to fix search functionality in sphinx1.8 (from readthedocs/sphinx_rtd_theme#672) (which is another reason for maintaining our own copy of this branch). The branch in this repository is now named version-dropdown-with-fixes (branching off of the version-dropdown branch in the sphinx_rtd_theme repository). In the long-term, I am a little concerned about using this theme that isn't showing any signs of being merged to the main branch of the readthedocs theme, but this has been working for us in other projects for the last 2 years, so I feel this is a reasonable approach in the short-medium term. The new process for building the documentation is given here: https://github.com/ESCOMP/CTSM/wiki/Directions-for-editing-CLM-documentation-on-github-and-sphinx Resolves ESCOMP#239

This changes the documentation theme to use the readthedocs theme, with some JavaScript that provides capabilities for a dropdown menu allowing you to select between different versions. This mimics the changes in ESMCI/cime#3439, which in turn was based on ESCOMP/CISM-wrapper#23.

billsacks added 6 commits December 9, 2018 05:47

Squashed 'doc/source/_themes/sphinx_rtd_theme/' content from commit b…

f755c7b

…46bba5 git-subtree-dir: doc/source/_themes/sphinx_rtd_theme git-subtree-split: b46bba51649fd15b90e6938b57a1e96f7063c81f

Merge commit 'f755c7b2e9e7efdc50a33ec4fa99d3beee2ca25e' as 'doc/sourc…

d052937

…e/_themes/sphinx_rtd_theme'

Update instructions for latest versioned_doc_builder

72e1dc6

- No longer have an -i argument (now just specify that via extra path elements in the -r argument) - Now have a versions subdirectory

billsacks added the documentation label Dec 9, 2018

billsacks self-assigned this Dec 9, 2018

billsacks merged commit 6fef0a9 into master Dec 9, 2018

billsacks mentioned this pull request Mar 6, 2020

WIP: Documentation: provide a version dropdown menu and change theme ESMCI/cime#3437

Closed

billsacks mentioned this pull request Mar 8, 2020

Documentation: provide a version dropdown menu and change theme ESMCI/cime#3439

Merged

billsacks mentioned this pull request Mar 30, 2020

Bring documentation source to master ESCOMP/CTSM#954

Merged

billsacks deleted the rtd_doc_theme branch November 11, 2020 21:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide a version drop-down menu in documentation #23

Provide a version drop-down menu in documentation #23

billsacks commented Dec 9, 2018

billsacks commented Dec 9, 2018

Provide a version drop-down menu in documentation #23

Provide a version drop-down menu in documentation #23

Conversation

billsacks commented Dec 9, 2018

billsacks commented Dec 9, 2018