Use sphinx-gallery to manage examples and tutorials (#268)

Enables the usage of sphinx-gallery with PyGMT. For now, requires features that aren't
yet released so we need to install from the Github master branch of sphinx-gallery.
Implements an image scrapper for `pygmt.Figure` that captures them and saves to the
correct file names. Need to add a global figure registry (`SHOWED_FIGURES` in
`pygmt/figure.py`) so that we can request all figure objects that have called their
`show` method. Replace the current jupyter notebooks by gallery notebook style `.py`
files, which are easier to maintain on version control.

.gitignore

@@ -2,19 +2,20 @@
 *~
 *.so
 *.pyd
-.pytest_cache/
-build/
-dist/
 .*.swp
+*.egg-info
 .coverage
 .cache
+.env
+MANIFEST
+.pytest_cache/
+.ipynb_checkpoints/
+build/
+dist/
 doc/_build
 doc/modules
+doc/gallery
+doc/tutorials
+doc/projections
 doc/api/gmt*.rst
-doc/api/generated
-.ipynb_checkpoints
-*.egg-info
-.env
-MANIFEST
-doc/tutorials/first-steps-*
-doc/tutorials/pygmt-*
+doc/api/generated

.travis.yml

@@ -121,6 +121,8 @@ before_install:
     # Download and install miniconda and setup dependencies
     # Need to source the script to set the PATH variable globaly
     - source continuous-integration/travis/setup-miniconda.sh
+    # Install the latest master of sphinx-gallery to use the scapper
+    - pip install --upgrade git+https://github.com/sphinx-gallery/sphinx-gallery.git
     # Show installed pkg information for postmortem diagnostic
     - conda list
 

CONTRIBUTING.md

@@ -37,6 +37,8 @@ read it carefully.
 * [How Can I Talk to You?](#how-can-i-talk-to-you)
 * [Reporting a Bug](#reporting-a-bug)
 * [Editing the Documentation](#editing-the-documentation)
+  - [Gallery plots](#gallery-plots)
+  - [Tutorials](#tutorials)
 * [Contributing Code](#contributing-code)
   - [General guidelines](#general-guidelines)
   - [Setting up your environment](#setting-up-your-environment)
@@ -104,6 +106,60 @@ download and install anything:
 Alternatively, you can make the changes offline to the files in the `doc` folder or the
 example scripts. See [Contributing Code](#contributing-code) for instructions.
 
+### Gallery plots
+
+The gallery and tutorials are managed by
+[sphinx-gallery](https://sphinx-gallery.readthedocs.io/)
+(currently, we need the development version from the Github master branch).
+The source files for the example gallery are `.py` scripts in `examples/gallery/` that
+generate one or more figures. They are executed automatically by sphinx-gallery when the
+documentation is built. The output is gathered and assembled into the gallery.
+
+You can **add a new** plot by placing a new `.py` file in one of the folders inside the
+`examples/gallery` folder of the repository. See the other examples to get an idea for the
+format.
+
+General guidelines for making a good gallery plot:
+
+* Examples should highlight a single feature/command. Good: *how to add a label to
+  a colorbar*. Bad: *how to add a label to the colorbar and use two different CPTs and
+  use subplots*.
+* Try to make the example as simple as possible. Good: use only commands that are
+  required to show the feature you want to highlight. Bad: use advanced/complex Python
+  features to make the code smaller.
+* Use a sample dataset from `pygmt.datasets` if you need to plot data. If a suitable
+  dataset isn't available, open an issue requesting one and we'll work together to add
+  it.
+* Add comments to explain things are aren't obvious from reading the code. Good: *Use a
+  Mercator projection and make the plot 6 inches wide*. Bad: *Draw coastlines and plot
+  the data*.
+* Describe the feature that you're showcasing and link to other relevant parts of the
+  documentation
+
+### Tutorials
+
+The tutorials (the User Guide in the docs) are also built by sphinx-gallery from the
+`.py` files in the `examples/tutorials` folder of the repository. To add a new tutorial:
+
+* Include a `.py` file in the `examples/tutorials` folder on the base of the repository.
+* Write the tutorial in "notebook" style with code mixed with paragraphs explaining what
+  is being done. See the other tutorials for the format.
+* Include the tutorial in the table of contents of the documentation (side bar). Do this
+  by adding a line to the User Guide `toc` directive in `doc/index.rst`. Notice that the
+  file included is the `.rst` generated by sphinx-gallery.
+
+Guidelines for a good tutorial:
+
+* Each tutorial should focus on a particular set of tasks that a user might want to
+  accomplish: plotting grids, interpolation, configuring the frame, projections, etc.
+* The tutorial code should be as simple as possible. Avoid using advanced/complex Python
+  features or abbreviations.
+* Explain the options and features in as much detail as possible. The gallery has
+  concise examples while the tutorials are detailed and full of text.
+
+Note that the `Figure.plot` function needs to be called for a plot to be inserted into
+the documentation.
+
 
 ## Contributing Code
 
@@ -282,6 +338,11 @@ Don't forget to commit the baseline image as well.
 
 ### Documentation
 
+> **NOTE**: We currently need the latest version of sphinx-gallery from the Github
+> master branch. Please install it with
+> `pip install --upgrade git+https://github.com/sphinx-gallery/sphinx-gallery.git`
+> before building the docs.
+
 Most documentation sources are in the `doc` folder.
 We use [sphinx](http://www.sphinx-doc.org/) to build the web pages from these sources.
 To build the HTML files:

Makefile

@@ -2,7 +2,7 @@
 PROJECT=pygmt
 TESTDIR=tmp-test-dir-with-unique-name
 PYTEST_ARGS=--cov-config=../.coveragerc --cov-report=term-missing --cov=$(PROJECT) --doctest-modules -v --pyargs
-BLACK_FILES=$(PROJECT) setup.py doc/conf.py
+BLACK_FILES=$(PROJECT) setup.py doc/conf.py examples
 FLAKE8_FILES=$(PROJECT) setup.py
 LINT_FILES=$(PROJECT) setup.py
 

doc/Makefile

@@ -26,8 +26,10 @@ clean:
 	rm -rf $(BUILDDIR)/linkcheck
 	rm -rf modules
 	rm -rf api/generated
+	rm -rf gallery
+	rm -rf tutorials
+	rm -rf projections
 	rm -rf .ipynb_checkpoints
-	rm -rf tutorials/first-steps-*
 
 html: api
 	@echo

doc/_templates/breadcrumbs.html

@@ -11,6 +11,16 @@
                 {% set body = "Please describe what could be improved about this page or the typo/mistake that you found:" %}
                 <a href="https://github.com/{{ github_repo }}/issues/new?title={{ title|urlencode }}&body={{ body|urlencode }}"
                    class="fa fa-github"> Improve this page</a>
+            {% elif pagename.split("/")[0] in galleries %}
+                {% set gallery_path = gallery_dir[pagename.split("/")[0]] %}
+                {% set example_script = pagename|replace(pagename.split("/")[0], gallery_dir[pagename.split("/")[0]]) %}
+                {% if pagename.split("/")[-1] == "index" %}
+                    {% set example_script = example_script|replace("index", "README.txt") %}
+                {% else %}
+                    {% set example_script = example_script + ".py" %}
+                {% endif %}
+                <a href="https://github.com/{{ github_repo }}/edit/{{ github_version }}/{{ doc_path }}/{{ example_script }}"
+                   class="fa fa-github"> Improve this page</a>
             {% else %}
                 <a href="https://github.com/{{ github_repo }}/edit/{{ github_version }}/{{ doc_path }}/{{ pagename }}{{ suffix }}"
                    class="fa fa-github"> Improve this page</a>

doc/api/index.rst

@@ -3,6 +3,8 @@
 API Reference
 =============
 
+.. automodule:: pygmt
+
 .. currentmodule:: pygmt
 
 Plotting
@@ -69,6 +71,10 @@ Miscellaneous
     print_clib_info
 
 
+.. automodule:: pygmt.datasets
+
+.. currentmodule:: pygmt
+
 Datasets
 --------
 
@@ -85,6 +91,10 @@ and store them in the GMT cache folder.
     datasets.load_japan_quakes
 
 
+.. automodule:: pygmt.exceptions
+
+.. currentmodule:: pygmt
+
 Exceptions
 ----------
 
@@ -102,6 +112,10 @@ All custom exceptions are derived from :class:`pygmt.exceptions.GMTError`.
     exceptions.GMTCLibNotFoundError
 
 
+.. automodule:: pygmt.clib
+
+.. currentmodule:: pygmt
+
 GMT C API
 ---------
 

doc/conf.py

@@ -1,9 +1,14 @@
 # -*- coding: utf-8 -*-
 import sys
 import os
+import glob
+import shutil
 import datetime
 import sphinx_rtd_theme
+import sphinx_gallery
+from sphinx_gallery.sorting import FileNameSortKey, ExplicitOrder
 from pygmt import __version__, __commit__
+from pygmt.sphinx_gallery import PyGMTScraper
 
 
 extensions = [
@@ -17,11 +22,9 @@
     "sphinx.ext.intersphinx",
     "numpydoc",
     "nbsphinx",
-    "pygmt.sphinxext.gmtplot",
+    "sphinx_gallery.gen_gallery",
 ]
 
-nbsphinx_allow_errors = True
-
 # Autosummary pages will be generated by sphinx-autogen instead of sphinx-build
 autosummary_generate = False
 
@@ -35,6 +38,34 @@
     "xarray": ("http://xarray.pydata.org/en/stable/", None),
 }
 
+sphinx_gallery_conf = {
+    # path to your examples scripts
+    "examples_dirs": [
+        "../examples/gallery",
+        "../examples/tutorials",
+        "../examples/projections",
+    ],
+    # path where to save gallery generated examples
+    "gallery_dirs": ["gallery", "tutorials", "projections"],
+    "subsection_order": ExplicitOrder(
+        ["../examples/gallery/coast", "../examples/gallery/plot"]
+    ),
+    # Patter to search for example files
+    "filename_pattern": r"\.py",
+    # Remove the "Download all examples" button from the top level gallery
+    "download_all_examples": False,
+    # Sort gallery example by file name instead of number of lines (default)
+    "within_subsection_order": FileNameSortKey,
+    # directory where function granular galleries are stored
+    "backreferences_dir": "api/generated/backreferences",
+    # Modules for which function level galleries are created.  In
+    # this case sphinx_gallery and numpy in a tuple of strings.
+    "doc_module": "pygmt",
+    # Insert links to documentation of objects in the examples
+    "reference_url": {"pygmt": None},
+    "image_scrapers": (PyGMTScraper(),),
+}
+
 # Sphinx project configuration
 templates_path = ["_templates"]
 exclude_patterns = ["_build", "**.ipynb_checkpoints"]
@@ -101,6 +132,10 @@
     # Custom variables to enable "Improve this page"" and "Download notebook"
     # links
     "doc_path": "doc",
+    "galleries": sphinx_gallery_conf["gallery_dirs"],
+    "gallery_dir": dict(
+        zip(sphinx_gallery_conf["gallery_dirs"], sphinx_gallery_conf["examples_dirs"])
+    ),
     "github_repo": "GenericMappingTools/pygmt",
     "github_version": "master",
 }

doc/index.rst

@@ -10,29 +10,6 @@
         </h2>
     </div>
 
-.. gmt-plot::
-    :center:
-
-    import pygmt
-
-    # Load sample earthquake data in a pandas.DataFrame
-    quakes = pygmt.datasets.load_usgs_quakes()
-
-    # Load the builtin Earth relief grid as an xarray.DataArray.
-    relief = pygmt.datasets.load_earth_relief(resolution="30m")
-
-    # The Figure object controls all plotting functions
-    fig = pygmt.Figure()
-    # Setup a map with a global region, a Mollweide projection, and automatic ticks
-    fig.basemap(region="g", projection="W200/8i", frame=True)
-    # Plot the Earth relief grid in pseudo-color.
-    fig.grdimage(relief, cmap="geo")
-    # Plot earthquakes as circles. Size maps to magnitude and color to depth.
-    fig.plot(x=quakes.longitude, y=quakes.latitude, sizes=0.01*2**quakes.mag,
-             color=quakes.depth/quakes.depth.max(), cmap="viridis", style="cc")
-    # Show a preview of the image (inline if in a Jupyter notebook).
-    fig.show()
-
 .. include:: ../README.rst
     :start-after: placeholder-for-doc-index
 
@@ -42,15 +19,18 @@
     :caption: Getting Started
 
     install.rst
-    tutorials/first-steps.ipynb
+    tutorials/overview.rst
+    gallery/index.rst
 
 .. toctree::
     :maxdepth: 2
     :hidden:
     :caption: User Guide
 
-    tutorials/plot-data-points.ipynb
-    sphinxext.rst
+    tutorials/frames.rst
+    projections/index.rst
+    tutorials/coastlines.rst
+    tutorials/plot.rst
 
 .. toctree::
     :maxdepth: 2