📝 Reworking SPFlow Docs #79

hayesall · 2020-06-04T19:23:45Z

Overview

This Pull Request builds on previous work in #31:

Significantly reworks the documentation to include tutorials from the README
- Fixes a deprecated sphinx build methods previously used in the Makefile
- Reorganizes into explicit source/build directories
- Tutorials are built with sphinx-gallery
Adds a new method in spn.io.Graphics for returning a pyplot instance (for rendering in examples or possibly in Jupyter notebooks)
- plot_spn(spn, fname) uses draw_spn(spn)
- _get_networkx_obj is now private since it was not used elsewhere in the repository
Adds SPFlow version information to a common location (_meta.py) for use in documentation / setup.py

Full details may be read from individual commit messages.

Limitations and Future Work

This does not document some of the GPU Examples (I do not fully know the assumptions SPFlow makes when torch/tensorflow does not find a GPU).
This did not include the "sampling" examples, since it was not clear to me which section they belong in (or whether they should be a separate topic).
This reworked the "SPFlow API" page to only include classes/methods specifically used in the tutorials. Many of these methods lack docstrings, so this could be a good place to start documenting function signatures.

Screenshots

This is a render of the home page:

Instead of using sphinx-apidoc to create a page for everything in the src/ directory, this only lists the signatures used in the tutorials (see the comment in "Limitations and Future Work" above).

The "Tutorials" Example Gallery reworks examples discussed in the project-level README. For example, this separates the examples by topic: "Syntax and DSL", "Tractable Probabilistic Models", and "Querying Probabilistic Models."

Files in the examples/ folder are built when the documentation builds, so the SPN shown on the page was the one learned from the SPN classification example.

Here is a screenshot showing the entire Tutorials page:

Add `__version__ = "0.0.40"` variable in `src/spn/_meta.py` Start pulling version info in `setup.py` from `_meta` The current documentation and `setup.py` had to be updated in two places, which could lead to inconsistencies. Introducing the variable here allows it to be updated once. There is some complexity in how `setup.py` accesses the __version__ variable. This is intended to prevent side-effects potentially caused by importing the library during setup, and is addressed in an in-line comment.

Add `docs/.gitignore` The alternative would be to add this to the project-level .gitignore at the base of the repository.

Add `docs/requirements.txt` Documentation has a specific set of dependencies which are independent of the rest of the library. This `requirements.txt` can be used to install these separately (e.g. in an automatic build system, or for a user who does not need to install the documentation libraries to use the code).

Drop deprecated `sphinx.main()` Start separating documentation source files into `docs/source/` Add `_static/.gitkeep` for folder persistence Add `_templates/.gitkeep` for folder persistence Add read from `_meta.py` to `conf.py` Drop commented-out code from `conf.py` Add `sphinx_gallery.gen_gallery` to `conf.py` extensions Basically: `sphinx.main()` was deprecated some time ago. It is possible to replicate some of its behavior using: ``` from sphinx.cmd.build import main ``` ... but this is technically a private API not intended for wide usage. `sphinx-build` should be more stable for the build process going forward. Replacing the `Makefile` also drops the use of multiple folders (e.g. `../src/spn/`, `../src/spn/experiments`), but it looks like these were excluded so they would not be part of a Public-API.

Rename `get_networkx_obj` -> `_get_networkx_obj` (private method) Refactor split `plot_spn(spn)` into two methods: - `draw_spn(spn)`: Returns a `pyplot` - `plot_spn(spn, fname="")`: Writes a `pyplot` to a file `_get_networkx_obj` was only used in the `spn.io.Graphics`, so its access could be reduced to private. `plot_spn()` was trying to do two things: draw an image and write it to disk. Splitting it into two methods allows each of these purposes to be used separately (e.g. if a user is working in a Jupyter environment and wants to visualize the SPN without opening a file browser).

Add `examples/README.txt` for `sphinx_gallery` Add `examples/language/README.txt` for syntax examples Add `examples/plot_spn_dsl` showing the DSL for SPNs Add `examples/plot_spn_object_hierarchy` using Sum/Products This reproduces some of the examples described in the README, but segments them into discrete tutorials meant to teach one thing at a time.

Move `usage/*` -> `source/usage/*` This move follows the pattern of separating source files into a source directory.

Add `examples/models/README.txt` Add `cnets.py` with Cutset Networks example Add `mixed_spn.py` with Mixed SPN example Add `multivariate_leaf.py` with Chow-Liu tree leaf example Add `parametric_spn.py` with Parametric SPN example Add `plot_learn_spn_classifier.py` with classification example This reproduces some of the points in the project README, but explicitly lists out some of the probabilistic models, allows them to be downloaded and written about individually, and visualizes some of the intermediate steps (for example, the `plot_learn_spn_classifier` plots the data, the learned SPN, and some of the results).

Add `examples/queries/README.txt` Add `plot_marginalize.py` showing marginal SPN example Add `plot_tractable_inference.py` showing marginal queries Add reference line to `plot_spn_object_hierarchy` These examples also pull from the project README, but are grouped so documentation about querying probabilistic models can be put in a common place and reasoned about individually.

Move `api.rst` -> `source/documentation/api.rst` Add `developing.rst` discussing notes on testing, docs Start enumearting functions/classes as needed This changes the API documentation quite a bit. The previous version would use `sphinx-apidoc` to generate pages for *everything* in the `src/` directory, which made it a little difficult to navigate and know what to look for. **Instead**, this lists the modules mentioned in the `examples/` and sorts them into whether they deal with "Structure," "Learning," "Inference," "Utilities," or "Datasets." Many of the modules are currently lacking docstrings, so listing modules in this manner could also help focus on which modules to document in the near future.

Move `index.rst` -> `source/index.rst` Add index overview based on the project README Add documentation structure This follows from the previous commits, and makes it so all new pages are built and rendered correctly.

Add `**Alexander L. Hayes** - *Indiana University, Bloomington*` ... following request for changes.

Add `.github/workflows/deploy_pages.yml` This action builds on the documentation rework in Pull Request SPFlow#79. This workflow: 1. Runs when changes are pushed to the `master` branch 2. Clones the repository 3. Sets up Python 4. Installs dependencies and runs `make html` 5. Deploys `docs/build/html/` on a `gh-pages` branch The GitHub repository settings may be configured to serve the contents of the `gh-pages` branch, so the main site will still be built from the `spflow.github.io` repository, and accessed at: - `https://spflow.github.io/` ... while the documentation will be served from: - `https://spflow.github.io/SPFlow/`

hayesall added 12 commits June 3, 2020 17:51

🙈 Adding docs/.gitignore for sphinx files

d7b5b32

Add `docs/.gitignore` The alternative would be to add this to the project-level .gitignore at the base of the repository.

🚚 Moving installation and quickstart

72f16f4

Move `usage/*` -> `source/usage/*` This move follows the pattern of separating source files into a source directory.

📝 Documentation index

0712506

Move `index.rst` -> `source/index.rst` Add index overview based on the project README Add documentation structure This follows from the previous commits, and makes it so all new pages are built and rendered correctly.

👌 Adding Alexander to README Contributors section

b284fb3

Add `**Alexander L. Hayes** - *Indiana University, Bloomington*` ... following request for changes.

hayesall mentioned this pull request Jun 7, 2020

👷 Documentation built through GitHub Actions #80

Merged

alejandromolinaml merged commit cfe7121 into SPFlow:master Jun 8, 2020

hayesall deleted the spflow_docs branch June 8, 2020 14:19

📝 Reworking SPFlow Docs #79

📝 Reworking SPFlow Docs #79

Conversation

hayesall commented Jun 4, 2020 • edited

Overview

Limitations and Future Work

Screenshots

hayesall commented Jun 4, 2020 •

edited