Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make documentation fully automatic #94

Merged
merged 22 commits into from Nov 22, 2021
Merged

Conversation

jtniehof
Copy link
Member

This is a substantial update to the docs handling to make sure the documentation is fully automatic: if new classes, methods, etc. are added all we need to do is add a docstring, not link it in anywhere by hand.

There are several changes here:

  • Remove patches for Sphinx 1.2 and set minimum Sphinx version to 1.3
  • Set up Sphinx to automatically document all members of a module. This is a combination of template updates and a few patches to Sphinx for earlier versions. Current versions of Sphinx work without a patch, so these can be gradually phased out.
  • Fix a deprecation warning for Sphinx 3 regarding the Javascript to hide the >>> in examples
  • Make sure every object has at least a basic docstring, with a one-line summary, so it is properly picked up in documentation.

There should be no functionality change (except removing a class that was never used.) This looks like a much bigger deal than it really is.

PR Checklist

  • Pull request has descriptive title
  • Pull request gives overview of changes
  • New code has inline comments where necessary
  • (N/A) Any new modules, functions or classes have docstrings consistent with dbprocessing style
  • (N/A) Major new functionality has appropriate Sphinx documentation
  • (N/A) Added an entry to release notes if fixing a major bug or providing a major new feature
  • (N/A) New features and bug fixes should have unit tests
  • (N/A) Relevant issues are linked in the description (use Closes # if this PR closes the issue, or some other reference, such as See # if it is related in some other way)

As there are no user-visible functionality changes, much of the above is irrelevant.

Documentation was tested to build on all minor versions of Sphinx from 1.3 through 4.2.

 * Patched in fixes for bugs in 1.2, which is now old enough to no
   longer be used.
 * Python 3 uses different naming convention for the contents of
   the build directory, so change the pathing so that Sphinx can
   find the dbprocessing package in that directory.
 * Bump to 1.3 for "needs" line; 1.2.4 doesn't exist, and 1.2.3 had
   bugs that were previously working around.
 * Handling of javascript mods for the html changed, wasn't going to
   work in 3.0. Basically copied fix from
   readthedocs/sphinx_rtd_theme#728
 * This was used for the "hide prompts" feature in example code.
   New version still works in 1.6, but no longer in 1.5 or earlier.
   Docs still render fine, just don't have the "hide" button, so this
   is okay.
 * autoinstanceattribute changed, so update class template
 * No special patching needed now, so tested on all released versions
   of sphinx, and remove check for maximum version supported.
 * Added in 072fafe but never used.
 * Just make sure there is at least a one-line at the top of each
   docstring so the autosummary table is populated properly. In
   a few cases docstrings were updated more specifically (particularly
   if there was none at all), but the goal is to get everything
   into the framework, not perfectly formatted.
@jtniehof jtniehof added the documentation Issues that can be addressed with docs (docstrings, sphinx, wiki) label Nov 18, 2021
@balarsen
Copy link
Contributor

My first look here is really good. One thing I have been experimenting with for other projects are the type annotations and sphinx really does a nice job with them.

e.g.

def myCoolFun(in1: ClassOne, in2: int, in3: np.ndarray) -> bool:
    #code

@jtniehof
Copy link
Member Author

Type notes are python 3 (relatively recent, at that), so will wait until after get out first release and start ripping out Python 2 support.

@dnadeau-lanl
Copy link
Contributor

I keep getting this error when building the docs. Circle CI does not show anything.
I am going to ignore it.

/projects/sdndev/scripts/common/dbprocessing/build/lib/dbprocessing/__init__.py:docstring of dbprocessing:7: WARNING: autosummary: failed to import reports
WARNING: autodoc: failed to import class 'runObj' from module 'dbprocessing.runMe'; the following exception was raised:
Traceback (most recent call last):
  File "/projects/sdndev/anaconda3/envs/dbprocessingUNH3/lib/python3.9/site-packages/sphinx/util/inspect.py", line 439, in safe_getattr
    return getattr(obj, name, *defargs)
AttributeError: module 'dbprocessing.runMe' has no attribute 'runObj'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/projects/sdndev/anaconda3/envs/dbprocessingUNH3/lib/python3.9/site-packages/sphinx/ext/autodoc/importer.py", line 110, in import_object
    obj = attrgetter(obj, mangled_name)
  File "/projects/sdndev/anaconda3/envs/dbprocessingUNH3/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 332, in get_attr
    return autodoc_attrgetter(self.env.app, obj, name, *defargs)
  File "/projects/sdndev/anaconda3/envs/dbprocessingUNH3/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 2780, in autodoc_attrgetter
    return safe_getattr(obj, name, *defargs)
  File "/projects/sdndev/anaconda3/envs/dbprocessingUNH3/lib/python3.9/site-packages/sphinx/util/inspect.py", line 455, in safe_getattr
    raise AttributeError(name) from exc
AttributeError: runObj

<rst_epilog>:2: WARNING: Document or section may not begin with a transition.
/projects/sdndev/scripts/common/dbprocessing/docs/index.rst:19: WARNING: autosummary: failed to import dbprocessing.reports
looking for now-outdated files... none found

Copy link
Contributor

@dnadeau-lanl dnadeau-lanl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In README.rst in the Helpful DB commands:, can you add delete from unixtime

@jtniehof
Copy link
Member Author

The reports import will fail if you don't have numpy or dateutil installed.

The "runObj" error suggests you didn't run python setup.py build before building the docs...it's finding an old version of runMe. runObj was removed in this PR.

"Document or section may not begin with a transition" is probably a symptom of one of the other issues. Those warnings/errors will probably result in an empty document to which the epilog is appended, meaning it'll start with the horizonal rule that is supposed to be between the main body and that footer. So see if that still stays around when the others are fixed.

You will probaby also want to do a make clean before building the docs, after taking care of the above.

Helpful DB commands is completely out of scope for this PR, which relates to the API documentation. I can add that later (maybe open an issue?) but I expect that particular section of the README to go away as the rest of the documentation is punched up.

@dnadeau-lanl
Copy link
Contributor

@jtniehof
Copy link
Member Author

I saw that your can specify 0 for ouput_product in PR #95. Can you update the doc?

  1. This PR is unrelated to No output fail in postgres #95
  2. The use of 0 for output_product isn't a behavior change. output_product can be any product_id. Whether that is a 0 or not depends on the autoincrement implementation of the DBMS. That test in No output fail in postgres #95 was to verify that the test for casting empty string to None did not destroy a 0 if that happened to be the ID of the output product. It's not clear if you're suggesting updating the docs on toNone (which isn't relevant, as 0 is not specially handled) or addProcess, where output_product is documented as the product ID, and an integer. That encompasses 0, and 0 isn't special.
  3. The scope of this PR is getting the structure set up, not content update. (There are many many docstrings that are definitely wrong.)

@dnadeau-lanl
Copy link
Contributor

make clean did the thing. I had left over .rst files in the autosummary directory.

@dnadeau-lanl
Copy link
Contributor

I did not realized that 0 was an id in your test. The test says no_output, I thought 0 was testing a no_output ID.

@dnadeau-lanl
Copy link
Contributor

I really like the docs. Just going through... I a might be a little to thorough for this PR. 😄

@jtniehof
Copy link
Member Author

I did not realized that 0 was an id in your test. The test says no_output, I thought 0 was testing a no_output ID.

Whoops, yep, that's confusing and deserved a comment at least, should have been in #95. I'll make a very small separate PR to clarify that.

@jtniehof
Copy link
Member Author

I really like the docs. Just going through... I a might be a little to thorough for this PR. smile

Maybe it's worth opening up an issue of just "pending docs updates" and throwing everything you find in there? I'll be doing a scrub shortly, so having a TODO list would definitely help.

@dnadeau-lanl dnadeau-lanl merged commit 7936301 into spacepy:master Nov 22, 2021
@jtniehof jtniehof deleted the docs_update branch November 22, 2021 22:00
@jtniehof jtniehof mentioned this pull request Feb 8, 2022
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Issues that can be addressed with docs (docstrings, sphinx, wiki)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants