What is this?
- A one hour Sphinx tutorial introduction.
- By Christoph Deil and Stuart Mumford .
- March 21, 2016 at PyAstro16 .
You will learn how to add Sphinx documentation to a Python package (using the
astrospam Python package in this repo).
The focus is exclusively on technical aspects how to work with Sphinx. We will not have time to talk about how to write good documentation, i.e. what content to create and how to structure it.
You are encouraged to follow along, i.e. try out every step on your computer after we demo it.
Before we start:
- Who has run Sphinx before (i.e. run
python setup.py build_sphinx)?
- Who has set up Sphinx before (i.e. run
If you have a question, or something isn't working for you, or if I'm going too fast, please feel free to interrupt us at any time!
We'll start with a quick overview of Sphinx and related things by having a look at the following web pages.
If you want to learn more, please go back and read the info on those pages after the tutorial on your own.
- We won't talk about this Sphinx. (I don't know why the Sphinx documentation generator was given that name.)
- To get some basic info on Sphinx, read the Wikipedia on Sphinx (documentation generator) or the welcome page of the Sphinx website at http://www.sphinx-doc.org/ .
- The most useful pages in the Sphinx documentation to get started are the Sphinx tutorial and the reStructuredText Primer
- Almost all Python projects use reStructured text (RST) and Sphinx for documentation. Examples: Python Astropy, Astroplan
- As the Wikipedia article on reStructuredText (RST)
explains, RST is a markup language (like LaTeX or Markdown) that is mostly used for Python docstrings (in
.pyfiles) and high-level documentation (in
- Sphinx is the tool that takes RST as input and produces HTML or PDF as output.
To be more precise, Sphinx is a Python package that is mostly used via the command line tools
sphinx-build(which again you typically invoke via a
- Python docstrings are extracted by the Sphinx "autodoc" feature to auto-generate API (application programming interface) docs. There's a few different formats for docstrings in use that Sphinx supports.
- The one all scientific Python packages (Numpy, Scipy, Astropy, ...) use is called the Numpy docstring standard which as added as a built-in Sphinx extension called sphinx.ext.napoleon (I don't know why it was called Napoleon.)
- Once documentation is set up for your package, it's typically easy to generate HTML
output by just running
make htmlwhich calls
sphinx-build, or by executing
python setup.py build_sphinxwhich runs the Sphinx build. Then you can look at the output by just opening up
index.htmlin some output directory where the HTML docs have been generated. Working on documentation is then a matter of editing
make htmland checking the HTML output or console for errors and warnings.
- Finally, if you want to host the generated HTML, the free https://readthedocs.org/ and https://pages.github.com/ services are good options. We won't have time to cover those today, feel free to ask us after if you want to learn how they work or want help to set it up for your project.
Let's go ahead with our hands-on introduction to Sphinx and start using it ...
Open a terminal and type
sphinx<TAB>. If this lists
sphinx-build), you have Sphinx installed.
sphinx-build --version to check the Sphinx version number.
The latest stable version is 1.3. If you have 1.2 or older, I'd suggest you update now e.g. using:
$ pip install --upgrade sphinx $ conda install sphinx
Later on we'll use the sphinx_rtd_theme . Please install it now via:
$ pip install sphinx_rtd_theme
Before we continue, everyone please check that you're set up:
$ sphinx-build --version Sphinx (sphinx-build) 1.3.6 $ python -c 'import sphinx_rtd_theme' # Should give no output. # If you get an ImportError, `sphinx_rtd_theme` isn't installed correctly.
Let's say you have a Python project consisting of a few
and would like to use Sphinx to generate HTML or PDF documentation for it.
As an example for today's tutorial, please grab this repo:
$ git clone https://github.com/cdeil/sphinx-tutorial $ cd sphinx-tutorial
As you can see, there is a Python package called
$ tree . . ├── LICENSE ├── README.rst └── astrospam ├── __init__.py ├── ham.py ├── pyastro16.py └── spam.py
But there's no HTML documentation for it. Let's change that!
To add Sphinx documentation, you run sphinx-quickstart
This will prompt you for some information and then generate a few of files.
For most questions you can just hit
ENTER to accept the default. These are
the questions where you don't take the default, but actually put something:
$ sphinx-quickstart Welcome to the Sphinx 1.3.6 quickstart utility. > Root path for the documentation [.]: docs > Project name: astrospam > Author name(s): Astrospam developers > Project version: 0.1 > autodoc: automatically insert docstrings from modules (y/n) [n]: y Finished: An initial directory structure has been created.
The tool created the following files:
docs/conf.py-- Sphinx configuration file (a Python file)
docs/index.rst-- Name of your master docs page (a reStructuredText, aka RST file)
docs/Makefile-- Makefile as convenience to run Sphinx (for Linux and Mac OS X)
docs/make.bat-- Makefile for Windows
And the following empty directories:
docs/_build-- This is where all output files (e.g. HTML) will go when Sphinx runs.
docs/_static-- A place for static files, e.g. images or css (we won't use it)
docs/_templates-- A place for template files (we won't use it)
Now we're all set up to generate HTML docs:
$ cd docs $ make html sphinx-build -b html -d _build/doctrees . _build/html Running Sphinx v1.3.6 making output directory... loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [html]: targets for 1 source files that are out of date updating environment: 1 added, 0 changed, 0 removed reading sources... [100%] index looking for now-outdated files... none found pickling environment... done checking consistency... done preparing documents... done writing output... [100%] index generating indices... genindex writing additional pages... search copying static files... done copying extra files... done dumping search index in English (code: en) ... done dumping object inventory... done build succeeded. Build finished. The HTML pages are in _build/html.
Now open up
_build/html/index.html in your webbrowser.
On Mac you can do:
$ open _build/html/index.html
Sphinx has generated a documentation webpage for you (with a sidebar, search field, main content area, footer)!
There's some other things you can do. Type
make help to find out:
$ make Please use `make <target>' where <target> is one of html to make standalone HTML files dirhtml to make HTML files named index.html in directories singlehtml to make a single large HTML file pickle to make pickle files json to make JSON files htmlhelp to make HTML files and a HTML help project qthelp to make HTML files and a qthelp project applehelp to make an Apple Help Book devhelp to make HTML files and a Devhelp project epub to make an epub latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter latexpdf to make LaTeX files and run them through pdflatex latexpdfja to make LaTeX files and run them through platex/dvipdfmx text to make text files man to make manual pages texinfo to make Texinfo files info to make Texinfo files and run them through makeinfo gettext to make PO message catalogs changes to make an overview of all changed/added/deprecated items xml to make Docutils-native XML files pseudoxml to make pseudoxml-XML files for display purposes linkcheck to check all external links for integrity doctest to run all doctests embedded in the documentation (if enabled) coverage to run coverage check of the documentation (if enabled)
If you have
pdflatex installed, you can try making a PDF version of your docs:
$ make latexpdf sphinx-build -b latex -d _build/doctrees . _build/latex Running Sphinx v1.3.6 making output directory... loading pickled environment... done building [mo]: targets for 0 po files that are out of date building [latex]: all documents updating environment: 0 added, 0 changed, 0 removed looking for now-outdated files... none found processing astrospam.tex... index resolving references... writing... done copying TeX support files... done build succeeded. Running LaTeX files through pdflatex... ... 1000 lines of horrible LaTeX log output ... Output written on astrospam.pdf (7 pages, 43725 bytes). Transcript written on astrospam.log. pdflatex finished; the PDF files are in _build/latex.
_build/latex/astrospam.pdf and have a look:
$ open _build/latex/astrospam.pdf
We're all set up to write some documentation ...
Now let's write some documentation.
This is done by adding text to
docs/index.rst, or by adding extra
text files in
docs and writing text using RST format there.
Writing documentation is a cycle similar to writing code:
- Check output HTML files
For the following exercises, and generally while learning how to write RST, it's very helpful to have the "reStructuredText Primer" page from the Sphinx docs open: http://www.sphinx-doc.org/en/stable/rest.html
Let's do the documentation writing cycle once:
index.rstand add this line after the title:
Refresh the browser and watch the text appear in the HTML output.
Add a sub-section called "Getting started".
Add a code example:
$ python >>> import astrospam >>> astrospam.spam() Spam Spam Spam >>> exit()
Add a sub-page
docs/tutorial.rstand copy & paste the following content there:
Tutorial ======== This is the ``astrospam`` tutorial. Part 1 ------ Spam, spam, spam ... Part 2 ------ More spam! https://youtu.be/anwy2MPT5RE
Now add that page to the toctree
.. toctree:: :maxdepth: 2 tutorial
Let's see what happens if we make an
RST formatting mistake.
Remove some underline characters from the title:
Welcome to astrospam's documentation! ============================
Sphinx will emit a warning pointing out the file and line number where the problem is and give a helpful message what the problem is:
docs/index.rst:7: WARNING: Title underline too short.
The HTML output will still be OK .. it's just a warning.
Let's see what happens if you make an error in a Sphinx directive.
E.g. you could change the
toctree directive in
.. toctreeeeeee:: :maxdepth: 2
Now you'll get an error an the TOC will be missing in the output.
Run these commands:
mkdir -p docs/_templates/autosummary wget https://raw.githubusercontent.com/astropy/package-template/a956da77759743b06db99d207b8e1e1a9eaf8a87/docs/_templates/autosummary/base.rst wget https://raw.githubusercontent.com/astropy/package-template/a956da77759743b06db99d207b8e1e1a9eaf8a87/docs/_templates/autosummary/class.rst wget https://raw.githubusercontent.com/astropy/package-template/a956da77759743b06db99d207b8e1e1a9eaf8a87/docs/_templates/autosummary/module.rst
TODO: link from docstrings to docs in RST file and the other way around.
Note that Sphinx autodoc imports the Python module and accesses
docstrings stored in
__doc__ attributes. This means that
module-level and class-level code is executed.
TODO: Illustrate by adding print statements.
TODO: Add code that throws an exception (e.g.
import spam or
1/0 or a
and show the resulting Sphinx error message.
Explain about __all__
TODO: show how to change to the readthedocs template and what changes.
7. Final comments
- We hope that this tutorial gave you a basic understanding of what Sphinx is, how it works, and how you use it to generate the documentation for Python projects.
- You should now be able to contribute to the documentation of existing Python projects and maybe even be able to set up Sphinx for your own package (e.g. by copy & pasting the working package-template setup).
- There's many things we didn't cover that will come up if you start contributing to Sphinx documentation for projects like Astropy or Astropy-affiliated packages: plot directive, setup.py integration, doctests, ...
- Sphinx, like other documentation generators such as LaTeX or Doxygen, is a very complicated, and extremely extensible and customisable tool. Even with years of experience you can easily get stuck with an uncomprehensible error message and get frustrated. Don't be shy to ask for help!