Example notebooks in Sphinx documentation (#941)

* started adding auto-build of notebooks in docs * reoganized API docs and fixed some build problems * cleaned up some notebookx * updated changelog (was very out of date) * remove the old notebooks from examples * added examples notesbooks (low-level) * exclude jupyter checkpoints * fixed style error and updated FAQ * move more examples to docs - made some modifications to table_writer_reader so that no cells fail (even ones that are supposed to) * implemented name changes requested * small cleanup * removed old execute_ipynb script deprecated - now uses nbsphinx * remove execute_ipynb from travis
cta-observatory · Jan 29, 2019 · bcee2ff · bcee2ff
1 parent c1f9931
commit bcee2ff
Show file tree

Hide file tree

Showing 81 changed files with 5,348 additions and 10,922 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -58,7 +58,6 @@ script:
     - pytest --cov=ctapipe
     - ctapipe-info --version # print out the version string
     - travis-sphinx -v --outdir=docbuild build --source=docs/
-    - ./execute_ipynbs.py
 
 after_script:
     - source deactivate

diff --git a/README.rst b/README.rst
@@ -22,7 +22,7 @@ expert or developer!
 
 * Code: https://github.com/cta-observatory/ctapipe
 * Docs: https://cta-observatory.github.io/ctapipe/
-* Example notebooks: https://github.com/cta-observatory/ctapipe/tree/master/examples/notebooks
+
 
 Installation for Users
 ----------------------

diff --git a/ctapipe/image/toymodel.py b/ctapipe/image/toymodel.py
@@ -15,10 +15,6 @@
     >>> print(image.shape)
     (400,)
 
-.. plot:: image/image_example.py
-    :include-source:
-
-
 
 """
 import numpy as np

diff --git a/docs/FAQ.rst b/docs/FAQ.rst
@@ -1,8 +1,35 @@
+==========================
 Frequently Asked Questions
 ==========================
 
+----------------
+Technical Issues
+----------------
+
+Missing standard library when compiling on macOS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you get an error when running `make develop` (or `python setup.py
+develop`) that says `fatal error: 'iostream' file not found`, this is likely
+because latest versions of macOS put the standard C++ libraries inside a
+package, rather than in /usr/lib (where older versions of setuptools expect
+them to be apparently). If this error occurs, you can just run the following
+command to re-create the /usr/lib links:
+
+.. code-block:: console
+
+    sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
+
+
 Hard crash when loading a SimTelArray file?
------------------------------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+*Note*: as of ctapipe 0.6.1, the new `SimTelEventSource` (based on `pyeventio`)
+is used by default instead of `HESSIOEventSource` (based on
+`pyhessio`/`libhessio`).  This should prevent any crashes as it does not use
+the `libhessio` library. If you are still using an older version, or have
+explicitly enabled the `HESSIOEventSource` rather than the new
+`SimTelEventSource`, see the following:
 
 Sometimes when reading a simtel file, the code crashes (not even an
 out-of-memory error is given). Loading a simtel file right now needs a
@@ -16,3 +43,4 @@ the mean time, you can increase your stack memory limit by using the
 commmand `ulimit -s <SIZE IN MB>` for a single terminal
 session. Increasing it above the default should help stop these random
 crashes.
+
diff --git a/docs/changelog.rst b/docs/changelog.rst
@@ -2,6 +2,67 @@
 Change Log
 ==========
 
+
+v0.6.1
+------
+
+* Fix broken build (#743) @kosack
+* Add example script for a simple event writer (#746) @jjlk
+* Fix camera axis alignment in HillasReconstructor (#741) @mackaiver
+* Lst reader (#749) @FrancaCassol
+* replace deprecated astropy broadcast (#754) @mackaiver
+* A few more example notebooks (#757) @kosack
+* Add MC xmax info (#759) @mackaiver
+* Use Astropy Coordinate Transofmations For Reconstruction (#758) @mackaiver
+* Trigger pixel reader (#745) @thomasarmstrong
+* Change requested in #742: init Hillas skewness and kurtosis to NaN (#744) @STSpencer
+* Fix call to np.linalg.leastsq (#760) @kosack
+* Fix/muon bugs (#762) @kosack
+* Implement hillas features usen eigh (#748) @MaxNoe
+* Use HillasParametersContainer only (#763) @MaxNoe
+* Regression features in `RegressorClassifierBase` (#764) @vuillaut
+* Adding an example notebook no how to convert hex geometry to square and back (#767) @vuillaut
+* Wrong angle in ArrayDisplay. changed phi to psi. (#771) @thomasgas
+* Unstructured interpolator (#770) @ParsonsRD
+* Lst reader (#776) @FrancaCassol
+* Fixing core reconstruction (#777) @kpfrang
+* Leakage (#783) @MaxNoe
+* Revert "Fixing core reconstruction" (#789) @kosack
+* Fixing the toy image generator (#790) @MaxNoe
+* Fix bad builds by changing channel name (missing pyqt package) (#793) @kosack
+* Implement concentration image features (#791) @MaxNoe
+* updated main documentation page (#792) @kosack
+* Impact intersection (#778) @mackaiver
+* add test for sliced geometries for hillas calculation (#781) @mackaiver
+* Simple HESS adaptations (#794) @ParsonsRD
+* added a config file for github release-drafter plugin (#795) @kosack
+* Array plotting (#784) @thomasgas
+* Minor changes: mostly deprecationwarning fixes (#787) @mireianievas
+* Codacy code style improvements (#796) @dneise
+* Add unit to h_max in HillasReconstructor (#797) @jjlk
+* speed up unit tests that use test_event fixture (#798) @kosack
+* Update Timing Parameters (#799) @LukasNickel
+
+v0.6.0
+------
+
+This is an interim release, after some major refactoring, and before we add
+the automatic gain selection and refactored container classes. It's not
+intended yet for production.
+
+Some Major changes since last release:
+
+* new `EventSource` class hierarchy for reading event data, which now supports simulation and testbench data from multiple camera prototypes (notably CHEC, SST-1M, NectarCam)
+* new `EventSeeker` class for (inefficient) random event access.
+* a much improved `Factory` class
+* re-organized event data structure (still evolving) - all scripts not in ctapipe must be changed to work with the new data items that were re-named  (a migration guide will be given in the 0.7 release)
+* better HDF5 table output, supporting merging multiple `Containers` into a single output table
+* improvements to Muon analysis, and the muon example script
+* improvements to the calibration classes
+* big improvements to the Instrument classes
+* lots of cleanups and bug fixes
+* much more...
+
 v0.5.3 (unlreleased)
 --------------------
 

diff --git a/docs/conf.py b/docs/conf.py
@@ -48,10 +48,13 @@
               'sphinx.ext.githubpages',
               'sphinx.ext.mathjax',
               'sphinx_automodapi.automodapi',
+              'nbsphinx',
               'matplotlib.sphinxext.plot_directive',
               'numpydoc']
 
 numpydoc_show_class_members = False
+nbsphinx_timeout = 120  # allow max 2 minutes to build each notebook
+
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -93,7 +96,7 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This patterns also effect to html_static_path and html_extra_path
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', '**.ipynb_checkpoints']
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = 'sphinx'
@@ -186,7 +189,7 @@
     'pandas': ('http://pandas.pydata.org/pandas-docs/stable/', None),
     'matplotlib': ('http://matplotlib.org/', None),
     'cython': ('http://docs.cython.org/en/latest/', None),
-    'iminuit': ('http://iminuit.readthedocs.io/en/latest/', None)
+    'iminuit': ('https://iminuit.readthedocs.io/en/latest/', None)
 }
 
 # on_rtd is whether we are on readthedocs.org

diff --git a/docs/analysis/index.rst → docs/ctapipe_api/analysis/index.rst b/docs/analysis/index.rst → docs/ctapipe_api/analysis/index.rst
diff --git a/docs/calib/index.rst → docs/ctapipe_api/calib/index.rst b/docs/calib/index.rst → docs/ctapipe_api/calib/index.rst
diff --git a/docs/calib/index_camera.rst → docs/ctapipe_api/calib/index_camera.rst b/docs/calib/index_camera.rst → docs/ctapipe_api/calib/index_camera.rst
diff --git a/docs/coordinates/index.rst → docs/ctapipe_api/coordinates/index.rst b/docs/coordinates/index.rst → docs/ctapipe_api/coordinates/index.rst
diff --git a/docs/core/corediagram.dot → docs/ctapipe_api/core/corediagram.dot b/docs/core/corediagram.dot → docs/ctapipe_api/core/corediagram.dot
diff --git a/docs/core/index.rst → docs/ctapipe_api/core/index.rst b/docs/core/index.rst → docs/ctapipe_api/core/index.rst
diff --git a/docs/core/tool-component.pdf → docs/ctapipe_api/core/tool-component.pdf b/docs/core/tool-component.pdf → docs/ctapipe_api/core/tool-component.pdf
diff --git a/docs/core/tool-component.png → docs/ctapipe_api/core/tool-component.png b/docs/core/tool-component.png → docs/ctapipe_api/core/tool-component.png
diff --git a/docs/image/charge_extractors.rst → docs/ctapipe_api/image/charge_extractors.rst b/docs/image/charge_extractors.rst → docs/ctapipe_api/image/charge_extractors.rst
diff --git a/docs/image/cleaning.rst → docs/ctapipe_api/image/cleaning.rst b/docs/image/cleaning.rst → docs/ctapipe_api/image/cleaning.rst
@@ -10,7 +10,7 @@ filtering).
 
 An example of image cleaning and dilation:
 
-.. image:: dilate.png
+.. image:: ./dilate.png
 
 
 API Reference

diff --git a/docs/image/dilate.png → docs/ctapipe_api/image/dilate.png b/docs/image/dilate.png → docs/ctapipe_api/image/dilate.png
diff --git a/docs/image/geometry_converter.rst → .../ctapipe_api/image/geometry_converter.rst b/docs/image/geometry_converter.rst → .../ctapipe_api/image/geometry_converter.rst
diff --git a/docs/image/hillas.rst → docs/ctapipe_api/image/hillas.rst b/docs/image/hillas.rst → docs/ctapipe_api/image/hillas.rst
diff --git a/docs/image/index.rst → docs/ctapipe_api/image/index.rst b/docs/image/index.rst → docs/ctapipe_api/image/index.rst
diff --git a/docs/image/muon.rst → docs/ctapipe_api/image/muon.rst b/docs/image/muon.rst → docs/ctapipe_api/image/muon.rst
diff --git a/docs/image/pixel_likelihood.rst → docs/ctapipe_api/image/pixel_likelihood.rst b/docs/image/pixel_likelihood.rst → docs/ctapipe_api/image/pixel_likelihood.rst
diff --git a/docs/image/reducers.rst → docs/ctapipe_api/image/reducers.rst b/docs/image/reducers.rst → docs/ctapipe_api/image/reducers.rst
diff --git a/docs/image/toymodel.rst → docs/ctapipe_api/image/toymodel.rst b/docs/image/toymodel.rst → docs/ctapipe_api/image/toymodel.rst
diff --git a/docs/ctapipe_api/index.rst b/docs/ctapipe_api/index.rst
@@ -0,0 +1,8 @@
+API Docs
+=====================
+
+.. toctree::
+  :maxdepth: 1
+  :glob:
+
+  */index
diff --git a/docs/instrument/camera.rst → docs/ctapipe_api/instrument/camera.rst b/docs/instrument/camera.rst → docs/ctapipe_api/instrument/camera.rst
@@ -87,7 +87,7 @@ with `numpy` operations, since it is quite speed-efficient.
 Examples
 --------
 
-.. plot:: instrument/camerageometry_example.py
+.. plot:: ctapipe_api/instrument/camerageometry_example.py
     :include-source:
 
 

diff --git a/docs/instrument/camerageometry_example.py → ..._api/instrument/camerageometry_example.py b/docs/instrument/camerageometry_example.py → ..._api/instrument/camerageometry_example.py
diff --git a/docs/instrument/config.dot → docs/ctapipe_api/instrument/config.dot b/docs/instrument/config.dot → docs/ctapipe_api/instrument/config.dot
diff --git a/docs/instrument/index.rst → docs/ctapipe_api/instrument/index.rst b/docs/instrument/index.rst → docs/ctapipe_api/instrument/index.rst
diff --git a/docs/instrument/optics.rst → docs/ctapipe_api/instrument/optics.rst b/docs/instrument/optics.rst → docs/ctapipe_api/instrument/optics.rst
diff --git a/docs/instrument/subarray.rst → docs/ctapipe_api/instrument/subarray.rst b/docs/instrument/subarray.rst → docs/ctapipe_api/instrument/subarray.rst
diff --git a/docs/instrument/telescope.rst → docs/ctapipe_api/instrument/telescope.rst b/docs/instrument/telescope.rst → docs/ctapipe_api/instrument/telescope.rst
diff --git a/docs/io/index.rst → docs/ctapipe_api/io/index.rst b/docs/io/index.rst → docs/ctapipe_api/io/index.rst
diff --git a/docs/io/shower.png → docs/ctapipe_api/io/shower.png b/docs/io/shower.png → docs/ctapipe_api/io/shower.png
diff --git a/docs/reco/ImPACT.rst → docs/ctapipe_api/reco/ImPACT.rst b/docs/reco/ImPACT.rst → docs/ctapipe_api/reco/ImPACT.rst
diff --git a/docs/reco/images/ImageTemplates.png → ...tapipe_api/reco/images/ImageTemplates.png b/docs/reco/images/ImageTemplates.png → ...tapipe_api/reco/images/ImageTemplates.png
diff --git a/docs/reco/images/goodness.png → docs/ctapipe_api/reco/images/goodness.png b/docs/reco/images/goodness.png → docs/ctapipe_api/reco/images/goodness.png
diff --git a/docs/reco/images/likelihood.png → docs/ctapipe_api/reco/images/likelihood.png b/docs/reco/images/likelihood.png → docs/ctapipe_api/reco/images/likelihood.png
diff --git a/docs/reco/images/predictions.png → docs/ctapipe_api/reco/images/predictions.png b/docs/reco/images/predictions.png → docs/ctapipe_api/reco/images/predictions.png
diff --git a/docs/reco/index.rst → docs/ctapipe_api/reco/index.rst b/docs/reco/index.rst → docs/ctapipe_api/reco/index.rst
diff --git a/docs/tools/index.rst → docs/ctapipe_api/tools/index.rst b/docs/tools/index.rst → docs/ctapipe_api/tools/index.rst
diff --git a/docs/utils/index.rst → docs/ctapipe_api/utils/index.rst b/docs/utils/index.rst → docs/ctapipe_api/utils/index.rst
diff --git a/docs/visualization/index.rst → docs/ctapipe_api/visualization/index.rst b/docs/visualization/index.rst → docs/ctapipe_api/visualization/index.rst