Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build TriBITS Documentation with Sphinx and deploy to the web #386

Closed
3 of 4 tasks
bartlettroscoe opened this issue Jun 27, 2021 · 34 comments
Closed
3 of 4 tasks

Build TriBITS Documentation with Sphinx and deploy to the web #386

bartlettroscoe opened this issue Jun 27, 2021 · 34 comments

Comments

@bartlettroscoe
Copy link
Member

bartlettroscoe commented Jun 27, 2021

CC: @marcinwrobel1986, @fnrizzi

Description

As discussed at the prior meeting on 6/24/2021, this Issue is to investigate the building and rendering of TriBITS documentation written in reStructuredText using Sphinx and deploy somehow to the web.

There are three documents that are generated using as reStructuredText files using the script:

$ cd TriBITS/
$ cd tribits/doc/
$ ./build_docs.sh

All the details of what gets built and how can be found in that script and the scripts and tools it calls.

To build the documentation locally, one needs to have docutils installed. But even without docutils installed, these scripts will still build everything needed to for the assembled files:

tribits/doc/guides/maintainers_guide/TribitsMaintainersGuide.rst
tribits/doc/guides/users_guide/TribitsUsersGuide.rst
tribits/doc/build_ref/TribitsBuildReference.rst

The script:

tribits/doc/publish_docs.sh

is used to publish the generated *html files on the tribits.org site (but you can run it locally to test it out).

Tasks

@bartlettroscoe
Copy link
Member Author

CC: @fnrizzi, @keitat

@marcinwrobel1986, as we discussed at last Thursday's meeting, this is the issue to investigate using Sphinx and readthedocs.com to improve the rendering of the TriBITS documentation currently hosted at tribits.org.

marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jun 30, 2021
- added script for setting up relative paths
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jun 30, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jun 30, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jun 30, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
- added options without numbering
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
marcinwrobel1986 added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 1, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe added a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe pushed a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
bartlettroscoe added a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
…#386)

This is not really the build reference for the TriBITS project per-say.  It
the build reference for an arbitrary TriBITS project <Project>.  It does not
make sense, I don't think, to produce this document for the TriBITS project
itself since it only has one package.  And people who are building the TriBITS
package and running TriBITS test will know quite a lot about TriBITS so I
think they can live with this generic document.
bartlettroscoe added a commit to bartlettroscoe/TriBITS that referenced this issue Jul 22, 2021
…st files (TriBITSPub#386)

You have to actually call the tool create-project-build-ref.py to generate the
final *.rst files.  You just need to call it and tell it to not generate any
of the other output (which is easy to do).
bartlettroscoe added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 23, 2021
You can't just run 'python3 sphinx_rst_generator.py' if you have already built
the documentation locally.  It will crash.  But note the warning at the top
about committing your work (or at least working in tracked non-ignored files).
bartlettroscoe added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 23, 2021
It is important to be able to rebuild the documentation iteratively locally in
a robust way.

The rebuild makes this an expensive tests by TriBITS standards (over 35s on my
super fast Linux machine) but this test is disabled by default and it only
takes one core to run so hopefully this is not too bad.
bartlettroscoe added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 23, 2021
I changed from 4-char to 2-char indents and wrapped lines to fit in 90 chars.
This really should be reformatted a little closer to 85 chars or so but at
least now I can do side-by-side editing on a reasonable side screen.

I know the Python standard is 4 char indents but if you do that you need a lot
more functions and a lot more wrapped lines to fit in 85 chars or so.
bartlettroscoe added a commit to bartlettroscoe/TriBITS that referenced this issue Jul 23, 2021
…it tested (TriBITSPub#386)

First, the function is_rst_file() no benefit from being a member of the class.
This is not Java.  Every function does not have to belong to a class.

Second, the function change_paths_and_get_includes() has a very clear purpose
and is unit testable but not when it was inside of the class.  Removing it
from the clas will allow it to be unit tested which will allow us to find and
fix a defect quickly.
bartlettroscoe added a commit to bartlettroscoe/TriBITS that referenced this issue Jul 23, 2021
…#386)

This shows the problem.  Now debuging the code and fixing it and ensuring it
stays fixed will be easy.
bartlettroscoe added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 23, 2021
…SPub#386)

This fixes a defect where loosing the indent was causing an error in the
converted *.rst file.
bartlettroscoe added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 23, 2021
bartlettroscoe added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 24, 2021
…ITSPub#386)

These includes got copies and pasted from another unit test driver.
bartlettroscoe added a commit to marcinwrobel1986/TriBITS that referenced this issue Jul 24, 2021
…riBITSPub#386)

When unit testing, you can't copy these files.  I have no idea how this unit
test passed locally when I ran it.  In production mode, the default is to copy
the files.  This is actually a pretty nice algorithm.

I also moved the unit test higher in the CMakeLists.txt file so this might
make it more likely that this unit test runs before the actual build of the
documentation.
bartlettroscoe added a commit that referenced this issue Jul 24, 2021
…entation-with-Sphinx-and-ReadTheDocs

Build and deploy TriBITS documentation with Sphinx (#386)

This seems a good time to deploy.
@bartlettroscoe
Copy link
Member Author

After merging #387, in this github repo's "Pages" settings, I sest:

image

and now you can see this site deployed at:

and the three documents are displayed there. Nice!

@bartlettroscoe
Copy link
Member Author

I updated the old tribits.org site to also point to the new Sphinx-generated HTML documents.

Lastly, I will post a GitHub Issue for the remaining issues for the Sphinx-generated documentation before we can switch over and abandon the DocUtils-generated documentation.

@bartlettroscoe
Copy link
Member Author

CC: @marcinwrobel1986

To follow on from this, I created #399 to address what needs to be done to drop the DocUtils-generated documents and fully transition over to Sphinx with no regressions. I also I created the follow-on issue #400 for taking full advantage of Sphinx after this transition to make these proper Sphinx/ReadTheDocs documents as people would expect.

NOTE: As part of the writing of #399 and a little more investigation, it seems that the formatting problems I am seeing with the Sphinx-generated documents is due to a problem with the sphinx and/or sphinx-rtd-theme installation on my SNL RHEL 7 machine (installed with pip3 install --local sphinx sphinx-rtd-theme). When the Sphinx documentation is built with the GitHub Actions job, the formatting problem I saw is gone. The problem was also gone when built using a sphinx and sphinx-rtd-theme installation on my Windows 10 laptop under Cygwin (using a Frankenstein setup mixing software installed for Cygwin and native Windows which is a miracle). (It is issues like this why I gave up on Sphinx years ago when I tried. It is just so time consuming fighting issues like this with complex software.)

I will now close this Issue and move on with the hard technical tasks in EPIC #367 where time is running out this FY. Hopefully there will be time and $$ in FY22 to complete #399 and #400 and complete this transition to the Sphinx.

Thanks for all of your hard work on this! What we have in place is pretty nice. (Just a bit more work and I think we would be completely done.)

TriBITS Refactor automation moved this from In Progress to Done Jul 27, 2021
@marcinwrobel1986
Copy link
Collaborator

Thank you Ross! I keep my fingers crossed for complete transition!

bartlettroscoe added a commit to bartlettroscoe/Trilinos that referenced this issue Sep 4, 2021
Brings in numerous refactorings to TriBITS over the last 3 months, but there
should be no breaks in backward compatibility.  Almost every file in TriBITS
is changed due to the lower-casing of command, macro and function names in PR
TriBITSPub/TriBITS#379.  But the main driver for this snapshot is to bring in
the change in PR TriBITSPub/TriBITS#413 that should make it so that Kokkos
INTERFACE_COMPILE_OPTIONS get propagated to downstream targets in TriBITS and
therefore to external customers through installed <Package>Config.cmake files
and IMPORTED targets.  I should have done several snapshots in the last few
months and not done a big snapshot like this (but I have been testing with
Trilinos locally along the way).

Overall, this merge brings in changes from a bunch of TriBITS PRs including
(from most recent):

* TriBITSPub/TriBITS#413: Change internal TriBITS target_link_libraries() to
  PUBLIC (TriBITSPub/TriBITS#299) component: core type: enhancement

* TriBITSPub/TriBITS#410: Upgrade from cmake 3.21.0 to 3.21.2
  (TriBITSPub/TriBITS#363, TriBITSPub/TriBITS#394)

* TriBITSPub/TriBITS#394: DO NOT MERGE: Show TriBITS test failures with CMake
  3.21.0 that don't occur with CMake 3.17.5 (TriBITSPub/TriBITS#363)

* TriBITSPub/TriBITS#409: Add getTestDictStatusField() to handle empty
  'status' field (SESW-383) component: ci_support type: enhancement

* TriBITSPub/TriBITS#408: Deal with spaces in CDash url parts (SESW-383)
  component: ci_support type: enhancement

* TriBITSPub/TriBITS#403: Spelling fixes

* TriBITSPub/TriBITS#407: Move tribits_get_build_url_and_write_to_file() to
  stand-alone module (TriBITSPub/TriBITS#154) component: ctest_driver type:
  enhancement

* TriBITSPub/TriBITS#388: Fixing typos (TriBITSPub/TriBITS#377)

* TriBITSPub/TriBITS#406: Fix tribits_ctest_driver() package-by-package mode
  for CMake 3.19+ (TriBITSPub/TriBITS#363, TriBITSPub/TriBITS#394) component:
  ctest_driver type: bug

* TriBITSPub/TriBITS#401: Improve GitHub Actions and CDash integration
  (TriBITSPub/TriBITS#154) type: enhancement

* TriBITSPub/TriBITS#366: CI: draft action yaml

* TriBITSPub/TriBITS#402: Revert some incorrect uppercase->lowercase changes

* TriBITSPub/TriBITS#387: Build and deploy TriBITS documentation with Sphinx
  (TriBITSPub/TriBITS#386) component: documentation type: enhancement

* TriBITSPub/TriBITS#398: Deal with pr null in not preprending build name
  (TriBITSPub/TriBITS#363) type: bug

* TriBITSPub/TriBITS#396: Send PR results to 'Pull Request' CDash group and
  add PR ID to build names (TriBITSPub/TriBITS#363) type: enhancement

* TriBITSPub/TriBITS#397: Print the ninja path and version
  (TriBITSPub/TriBITS#363)

* TriBITSPub/TriBITS#393: GitHub Actions based testing for TriBITS
  (TriBITSPub/TriBITS#363) type: enhancement

* TriBITSPub/TriBITS#389: TriBITS CI testing with GitHub Actions
  (TriBITSPub/TriBITS#363) type: enhancement

* TriBITSPub/TriBITS#392: Fix broken tests for non-Fortran and CMake 3.21
  builds (TriBITSPub/TriBITS#363) component: core

* TriBITSPub/TriBITS#391: Fix up build_docs.sh for Sphinx doc generation
  (TriBITSPub/TriBITS#386) component: documentation type: enhancement

* TriBITSPub/TriBITS#390: Add test for doc generation and fix usage of Python3
  component: documentation type: bug

* TriBITSPub/TriBITS#385: Replace last few references to
  TribitsDevelopersGuide.html (TriBITSPub/TriBITS#381) component:
  documentation type: enhancement

* TriBITSPub/TriBITS#384: Split TribitsDevelopersGuide.* into
  TribitsUsersGuide.* and TribitsMaintainersGuide.* (TriBITSPub/TriBITS#381)
  component: documentation type: enhancement

* TriBITSPub/TriBITS#383: Remove endfunction(<string>) and endmacro(<string>)
  (TriBITSPub/TriBITS#274, TriBITSPub/TriBITS#382) component: common_tpls
  type: bug

* TriBITSPub/TriBITS#380: More package-arch data-structure documentation
  updates (TriBITSPub/TriBITS#63) component: documentation type: enhancement

* TriBITSPub/TriBITS#379: Convert TriBITS to lower-case CMake command, macro,
  and function names (TriBITSPub/TriBITS#274)

The following files were conflicting where I went with what is on the Trilinos
'develop' branch:

* cmake/tribits/common_tpls/FindTPLBLAS.cmake
* cmake/tribits/common_tpls/FindTPLLAPACK.cmake
* cmake/tribits/common_tpls/FindTPLNetcdf.cmake

(It looks like the above changes never made it back into TriBITS proper.  The
conflicts were due to the case changes in cmake command calls in these files
due to TriBITSPub/TriBITS#379.)

There was also a conflict in the file:

* cmake/tribits/core/installation/TribitsProjectConfigTemplate.cmake.in

I looked at that one carefully and I think that may have been due to fixes on
both sides and then the case changes from TriBITSPub/TriBITS#379.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

2 participants