Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃摑 Reworking SPFlow Docs #79

Merged
merged 12 commits into from Jun 8, 2020
Merged

Conversation

hayesall
Copy link
Contributor

@hayesall hayesall commented Jun 4, 2020

Overview

This Pull Request builds on previous work in #31:

  1. Significantly reworks the documentation to include tutorials from the README
    • Fixes a deprecated sphinx build methods previously used in the Makefile
    • Reorganizes into explicit source/build directories
    • Tutorials are built with sphinx-gallery
  2. Adds a new method in spn.io.Graphics for returning a pyplot instance (for rendering in examples or possibly in Jupyter notebooks)
    • plot_spn(spn, fname) uses draw_spn(spn)
    • _get_networkx_obj is now private since it was not used elsewhere in the repository
  3. Adds SPFlow version information to a common location (_meta.py) for use in documentation / setup.py

Full details may be read from individual commit messages.

Limitations and Future Work

  • This does not document some of the GPU Examples (I do not fully know the assumptions SPFlow makes when torch/tensorflow does not find a GPU).
  • This did not include the "sampling" examples, since it was not clear to me which section they belong in (or whether they should be a separate topic).
  • This reworked the "SPFlow API" page to only include classes/methods specifically used in the tutorials. Many of these methods lack docstrings, so this could be a good place to start documenting function signatures.

Screenshots

This is a render of the home page:

index_page


Instead of using sphinx-apidoc to create a page for everything in the src/ directory, this only lists the signatures used in the tutorials (see the comment in "Limitations and Future Work" above).

SPFlow_api


The "Tutorials" Example Gallery reworks examples discussed in the project-level README. For example, this separates the examples by topic: "Syntax and DSL", "Tractable Probabilistic Models", and "Querying Probabilistic Models."

example_gallery


Files in the examples/ folder are built when the documentation builds, so the SPN shown on the page was the one learned from the SPN classification example.

learn_classification


Here is a screenshot showing the entire Tutorials page:

all_tutorials

Add `__version__ = "0.0.40"` variable in `src/spn/_meta.py`
Start pulling version info in `setup.py` from `_meta`

The current documentation and `setup.py` had to be updated
in two places, which could lead to inconsistencies. Introducing
the variable here allows it to be updated once.

There is some complexity in how `setup.py` accesses the
__version__ variable. This is intended to prevent side-effects
potentially caused by importing the library during setup,
and is addressed in an in-line comment.
Add `docs/.gitignore`

The alternative would be to add this to the project-level
.gitignore at the base of the repository.
Add `docs/requirements.txt`

Documentation has a specific set of dependencies which are
independent of the rest of the library. This `requirements.txt`
can be used to install these separately (e.g. in an automatic
build system, or for a user who does not need to install
the documentation libraries to use the code).
Drop deprecated `sphinx.main()`
Start separating documentation source files into `docs/source/`
Add `_static/.gitkeep` for folder persistence
Add `_templates/.gitkeep` for folder persistence
Add read from `_meta.py` to `conf.py`
Drop commented-out code from `conf.py`
Add `sphinx_gallery.gen_gallery` to `conf.py` extensions

Basically: `sphinx.main()` was deprecated some time ago. It is
possible to replicate some of its behavior using:

```
from sphinx.cmd.build import main
```

... but this is technically a private API not intended for
wide usage. `sphinx-build` should be more stable for the
build process going forward.

Replacing the `Makefile` also drops the use of multiple
folders (e.g. `../src/spn/`, `../src/spn/experiments`), but
it looks like these were excluded so they would not be
part of a Public-API.
Rename `get_networkx_obj` -> `_get_networkx_obj` (private method)
Refactor split `plot_spn(spn)` into two methods:
  - `draw_spn(spn)`: Returns a `pyplot`
  - `plot_spn(spn, fname="")`: Writes a `pyplot` to a file

`_get_networkx_obj` was only used in the `spn.io.Graphics`, so
its access could be reduced to private.

`plot_spn()` was trying to do two things: draw an image and
write it to disk. Splitting it into two methods allows each
of these purposes to be used separately (e.g. if a user is
working in a Jupyter environment and wants to visualize the
SPN without opening a file browser).
Add `examples/README.txt` for `sphinx_gallery`
Add `examples/language/README.txt` for syntax examples
Add `examples/plot_spn_dsl` showing the DSL for SPNs
Add `examples/plot_spn_object_hierarchy` using Sum/Products

This reproduces some of the examples described in the README,
but segments them into discrete tutorials meant to teach one
thing at a time.
Move `usage/*` -> `source/usage/*`

This move follows the pattern of separating source files into a
source directory.
Add `examples/models/README.txt`
Add `cnets.py` with Cutset Networks example
Add `mixed_spn.py` with Mixed SPN example
Add `multivariate_leaf.py` with Chow-Liu tree leaf example
Add `parametric_spn.py` with Parametric SPN example
Add `plot_learn_spn_classifier.py` with classification example

This reproduces some of the points in the project README, but
explicitly lists out some of the probabilistic models,
allows them to be downloaded and written about individually,
and visualizes some of the intermediate steps (for example,
the `plot_learn_spn_classifier` plots the data, the learned
SPN, and some of the results).
Add `examples/queries/README.txt`
Add `plot_marginalize.py` showing marginal SPN example
Add `plot_tractable_inference.py` showing marginal queries
Add reference line to `plot_spn_object_hierarchy`

These examples also pull from the project README, but are grouped
so documentation about querying probabilistic models can be
put in a common place and reasoned about individually.
Move `api.rst` -> `source/documentation/api.rst`
Add `developing.rst` discussing notes on testing, docs
Start enumearting functions/classes as needed

This changes the API documentation quite a bit. The previous
version would use `sphinx-apidoc` to generate pages for
*everything* in the `src/` directory, which made it a little
difficult to navigate and know what to look for. **Instead**,
this lists the modules mentioned in the `examples/` and sorts
them into whether they deal with "Structure," "Learning,"
"Inference," "Utilities," or "Datasets."

Many of the modules are currently lacking docstrings, so
listing modules in this manner could also help focus on
which modules to document in the near future.
Move `index.rst` -> `source/index.rst`
Add index overview based on the project README
Add documentation structure

This follows from the previous commits, and makes it so all
new pages are built and rendered correctly.
Add `**Alexander L. Hayes** - *Indiana University, Bloomington*`

... following request for changes.
hayesall added a commit to hayesall/SPFlow that referenced this pull request Jun 7, 2020
Add `.github/workflows/deploy_pages.yml`

This action builds on the documentation rework in
Pull Request SPFlow#79. This workflow:

1. Runs when changes are pushed to the `master` branch
2. Clones the repository
3. Sets up Python
4. Installs dependencies and runs `make html`
5. Deploys `docs/build/html/` on a `gh-pages` branch

The GitHub repository settings may be configured to
serve the contents of the `gh-pages` branch, so the
main site will still be built from the
`spflow.github.io` repository, and accessed at:

- `https://spflow.github.io/`

... while the documentation will be served from:

- `https://spflow.github.io/SPFlow/`
@alejandromolinaml alejandromolinaml merged commit cfe7121 into SPFlow:master Jun 8, 2020
@hayesall hayesall deleted the spflow_docs branch June 8, 2020 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants