[WIP] Improving Python Docs API #16392

sojiadeshina · 2019-10-08T00:58:13Z

Description

Reorganizes python API docs in the new website. We now have coverage on all the mxnet python submodules, building on #16377

@ThomasDelteil @aaronmarkham @eric-haibin-lin

Changed the API front page to provide quick links for frequently used packages, this is also reflected on the top level of the toc-tree in the left hand side.

There's also a new page that allows you to easily navigate to all the submodules.

…raries

aaronmarkham

Looks like you might need to rebase to get past the sanity check on CI.
Also, I have a bunch of questions.
This is a big change. Let's get it published on the staging server.

aaronmarkham · 2019-10-09T00:59:44Z

docs/python_docs/_static/autodoc.js

+        setTimeout(auto_index, 500);
+    }
+}
+$(document).ready(auto_index);


Is the no new line going to be a problem? Seems like I've failed some linter because of that before.

Linter didn't yell at me for this so I guess it's fine

aaronmarkham · 2019-10-09T01:01:22Z

docs/python_docs/_static/autodoc.js

+
+/* Customizations to the Sphinx auto module plugin output */
+function auto_index() {
+    var targets = $("dl.class>dt,dl.function>dt");


Can this function be described?
We ran into issues with the last site because the Sphinx customizations and js were undocumented.

Sure.
Essentially what's going on is it's going through all the classes/methods on the page and truncating the name to just the last two parts i.e from something like mxnet.gluon.nn.contrib to just nn.contrib and then adding that to a list that can be toggled so that the sidebar has information and anchors to specific methods on the page even when the sphinx toc-tree doesn't.

@ThomasDelteil wrote this part so he can correct me if i missed anything or give a more detailed description of what's going on.

aaronmarkham · 2019-10-09T01:02:58Z

docs/python_docs/environment.yml

@@ -36,3 +36,4 @@ dependencies:
  - breathe
  - mock
  - awscli
+  - autodocsumm


Is that from here? https://github.com/Chilipp/autodocsumm

yeah that's where it's from.

aaronmarkham · 2019-10-09T01:04:47Z

docs/python_docs/python/Makefile

@@ -47,7 +46,7 @@ build/%: %
 html: $(OBJ)
 	mkdir -p build
 	cp Makefile_sphinx build/Makefile
-	sphinx-autogen build/api/*.rst build/api/*/*.rst   -t build/_templates/
+	sphinx-autogen build/api/*.rst build/api/**/*.rst   -t build/_templates/


Is that ** some recursive notation for file paths?

That's the idea but actually I think the effect is the same with a single * .

aaronmarkham · 2019-10-09T01:08:38Z

docs/python_docs/python/api/gluon/model_zoo/index.rst


 .. automodule:: mxnet.gluon.model_zoo.vision
+   :noindex:


What is the reason on when you use this approach:

.. automodule:: mxnet.gluon.model_zoo.vision :noindex:

...versus this:

.. automodule:: mxnet.gluon.loss :members: :imported-members: :autosummary:

The difference is this.

For model_zoo.vision, I wanted to keep the documentation organized by the different model architectures: resnet, vgg etcs. it would have been super cluttered if it was just done all automatically. There are a ton of pretrained models in the model zoo so it's worth it to have them organized by sections.

The second approach in mxnet.gluon.loss automatically generates the documentation for all classes in the package and :autosummary: allows you to create a summary table at top automatically. Since there are not that many loss functions the autogenerated table summary is sufficient for this class.

aaronmarkham · 2019-10-09T01:09:33Z

docs/python_docs/python/api/gluon/contrib/index.rst

@@ -146,27 +147,21 @@ API Reference

 .. automodule:: mxnet.gluon.contrib
    :members:
-    :imported-members:


When do you decide to not include :imported-members:?

I take out imported members when the package level module also incidentally imports stuff from elsewhere that's already documented. For example here I think this also imports mxnet.ndarray which leads to a ton of warnings during the sphinx build.

I take out imported members when the package level module also incidentally imports stuff from elsewhere that's already documented. For example here I think this also imports mxnet.ndarray which leads to a ton of warnings during the sphinx build.

Ok that's good then, because we need to get warnings back down to zero.

aaronmarkham · 2019-10-09T01:11:36Z

docs/python_docs/python/api/gluon/nn/index.rst



 Sequential containers
 ---------------------


 .. autosummary::
-    :toctree: _autogen
    :nosignatures:


This was questioned in another issue. What are some set with :nosignatures: and others are not?

This is an aesthetic/readability choice and the idea here is similar to the comment in the other issue. Here the doc is listing different classes under the different groupings so there's no need to have the method signatures.

At the bottom when autosummary generates an alphabetical listing of the classes, the method signatures are included.

aaronmarkham · 2019-10-09T01:15:03Z

docs/python_docs/python/api/initializer/index.rst

+=================
+
+.. automodule:: mxnet.initializer
+    :members:


Why no inherited-members?

do you mean :imported-members:? if so then it's the same as : #16392 (comment)

if you meant :inherited-members: I think that only works for .. autoclass::

do you mean :imported-members:? if so then it's the same as : #16392 (comment)

if you meant :inherited-members: I think that only works for .. autoclass::

I noticed that sometimes it's imported-members and sometimes it's inherited-members. I would like to better understand the distinction and when one should be used, both should be used, none should be used.

aaronmarkham · 2019-10-09T20:52:08Z

@sojiadeshina Once this PR goes in you can publish to staging. Plus, I'm having the same sanity test issue even after rebasing, so you might want track that. #16411

ThomasDelteil · 2019-10-10T06:27:38Z

@ChaiBapchya please restart CI

ChaiBapchya · 2019-10-10T17:49:37Z

Retriggered the windows-gpu, windows-cpu

ChaiBapchya · 2019-10-10T21:39:05Z

@ThomasDelteil
@aaronmarkham @eric-haibin-lin Ready to review/merge

aaronmarkham

🚀

* doc fixes for gluon api sub modules only * update to the docs * adding the autodoc js plugin * adding autodoc javascript, fixing some warnings * fixing model zoo * python api docs major refactor -> ndarray, symbol and other mxnet libraries * restructure top level toctree navs and bubble more submodules to the top * modify import to reference defined classes and functions directly * fix some broken links * add extra classes imported from c++ * fix .htaccess redirects since api links have changed again * included license on files missing license * Update monitor.py

sojiadeshina and others added 9 commits October 7, 2019 17:36

doc fixes for gluon api sub modules only

8de689a

update to the docs

24328d0

adding the autodoc js plugin

7e1aa06

adding autodoc javascript, fixing some warnings

2c6b028

fixing model zoo

a973025

python api docs major refactor -> ndarray, symbol and other mxnet lib…

f5cdbf2

…raries

restructure top level toctree navs and bubble more submodules to the top

8d9ad74

modify import to reference defined classes and functions directly

c7c5a6e

fix some broken links

eb0ed2b

sojiadeshina requested review from aaronmarkham and szha as code owners October 8, 2019 00:58

sojiadeshina mentioned this pull request Oct 8, 2019

[WIP] Fixing the API organization #16377

Closed

sojiadeshina added 3 commits October 8, 2019 13:13

add extra classes imported from c++

14c2af0

fix .htaccess redirects since api links have changed again

bd4e2fa

included license on files missing license

651c7cc

aaronmarkham reviewed Oct 9, 2019

View reviewed changes

aaronmarkham mentioned this pull request Oct 10, 2019

Beta build #16411

Merged

Update monitor.py

608b5d5

ChaiBapchya mentioned this pull request Oct 10, 2019

[Website] Function parameters display inconsistent #16350

Closed

ThomasDelteil approved these changes Oct 10, 2019

View reviewed changes

aaronmarkham approved these changes Oct 10, 2019

View reviewed changes

aaronmarkham merged commit 243ade9 into apache:master Oct 10, 2019

sojiadeshina mentioned this pull request Oct 15, 2019

fix missing docs due to git add issues #16496

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Improving Python Docs API #16392

[WIP] Improving Python Docs API #16392

sojiadeshina commented Oct 8, 2019

aaronmarkham left a comment

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

sojiadeshina Oct 9, 2019

aaronmarkham Oct 9, 2019

aaronmarkham commented Oct 9, 2019

ThomasDelteil commented Oct 10, 2019

ChaiBapchya commented Oct 10, 2019

ChaiBapchya commented Oct 10, 2019

aaronmarkham left a comment

[WIP] Improving Python Docs API #16392

[WIP] Improving Python Docs API #16392

Conversation

sojiadeshina commented Oct 8, 2019

Description

aaronmarkham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronmarkham commented Oct 9, 2019

ThomasDelteil commented Oct 10, 2019

ChaiBapchya commented Oct 10, 2019

ChaiBapchya commented Oct 10, 2019

aaronmarkham left a comment

Choose a reason for hiding this comment