[docs] 🎨 Sphinx Autosummary for generating Python-API documentation

[hayesall]

Executive Summary

This replaces autoclass/autofunction with sphinx autosummary to create and paginate the Python API.

Summary of Changes

• Add docs/.gitignore to not track autosummary stubs
• Add sphinx.ext.autosummary in docs/conf.py
  • Add 'members' and 'inherited-members' as default parameters
  • Add 'autosummary = True' for setting output with :toctree:
• Add .. autosummary:: tags to replace .. autoclass:: and .. autofunction::

Pull Request Motivation 📝

Previously the Python-API.rst dumped all of the Python API onto a single page when sphinx-build generated html pages.

This replaces the Python-API documentation with an index listing all modules, and paginates all functions and classes onto separate pages.

In summary, this pull request:

• Requires less scrolling on the Python-API page
• Makes it easier to find specific pieces of the full Python documentation

Alternatives

This could also be accomplished by creating individual pages by hand and putting autoclass and autofunction on each. Autosummary simply makes this process more scalable.

Any feedback would be appreciated. Thank you developers!

---

Screenshots

Before (left) and After (right) of the Python-API.html landing page

Training API: lightgbm.cv. The lightgbm.cv function has its own page.

[msftclas]

All CLA requirements met.

[StrikerRUS]

@hayesall Thanks a lot for your contribution! It looks nice from a first glance! Please let me describe a few drawbacks I see.

• It breaks previous links to Python API from third-party sites.

• It generates duplicated entries for class constructors. I guess a workaround can be found by playing around with autoclass_content value:
  https://github.com/microsoft/LightGBM/blob/5d3a3ea47eb8693c924ffb891387e93534f92896/docs/conf.py#L120
  

  UPD: It can be fixed by setting autoclass_content to 'class'.

• It removes class inheritance (previously controlled by :show-inheritance: flag).
  
  UPD: Can be fixed by appending 'show-inheritance' to autodoc_default_flags.

[hayesall]

✅ Fixed the 'show-inheritance by adding it to the autodoc_default_flags variable:

autodoc_default_flags = ['members', 'inherited-members', 'show-inheritance']

---

✅ Changing autoclass_content to 'class' removes the duplicate __init__ entries.

---

❓ I don't have a solution for third-party links, but here is what I have currently:

• Existing links to a specific portion (e.g. Python-API.html#lightgbm.LGBMModel) will still direct users to the index (in this case: Python-API.html) rather than a 404 error page.
• ReadTheDocs has a special case for handling redirect links, but these are handled through their website rather than configuration files.
• Some plugins support JavaScript redirects, but would introduce another dependency.

[StrikerRUS]

@hayesall

Existing links to a specific portion (e.g. Python-API.html#lightgbm.LGBMModel) will still direct users to the index (in this case: Python-API.html) rather than a 404 error page.

Indeed! Since index page is rather small now, it should be not so hard for users to seek for the specified API member manually.

[StrikerRUS]

LGTM!

@hayesall Thank you for your contribution! Separate pages look much better than previous huge wall of API code.

[StrikerRUS]

@hayesall I'm sorry for the late question, but according to the logs it seems that autosummary scans each .rst file to generate a code. Is it possible to help it and specify that we need to scan only Python-API.rst? I guess it will save some time.

D:\Users\nekit\Downloads\LightGBM\docs>make html
Running Sphinx v1.8.5
making output directory...
[autosummary] generating autosummary for: Advanced-Topics.rst, C-API.rst, Development-Guide.rst, Experiments.rst, FAQ.rst, Features.rst, GPU-Performance.rst, GPU-Targets.rst, GPU-Tutorial.rst, GPU-Windows.rst, Installation-Guide.rst, Parallel-Learning-Guide.rst, Parameters-Tuning.rst, Parameters.rst, Python-API.rst, Python-Intro.rst, Quick-Start.rst, README.rst, gcc-Tips.rst, index.rst

...

[hayesall]

@StrikerRUS Sphinx's generate.py#L193-L228 module looks like it should support only scanning a specific list of files:

The autosummary_generate variable looks to be where this behavior is accessed, but I would need to experiment a bit to be sure.

I can look later today. If it's an easy fix I'll try to patch it in with #2293. If it's a bit more involved I'll open an issue.

[StrikerRUS]

@hayesall Thanks a lot for your answer with starting points! It's absolutely not urgent and critical.

Please do not include any fix in #2293, because that PR is about different things. Just post updates here if you find anything valuable or make a new PR if you'd like (and have enough free time 😄 ).