Merge pull request #4 from ericpre/add_docs_action

Improve documentation
hyperspy · Jul 25, 2022 · bb90340 · bb90340
2 parents 983caa4 + 9d6e6d6
commit bb90340
Show file tree

Hide file tree

Showing 35 changed files with 198 additions and 77 deletions.
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
@@ -3,8 +3,9 @@ name: Docs
 on: [push, pull_request]
 
 jobs:
-  build_doc:
-    name: Build docs
+  build_doc_PR:
+    # This build is to automatically add comments on PR
+    name: PR comments
     runs-on: ubuntu-latest
     env:
       BUILD_DEPS: python3-dev build-essential graphviz
@@ -23,3 +24,100 @@ jobs:
         with:
           path: ./docs/_build/html/
           name: docs_build
+
+  build_docs:
+    # This build is to check links and update table of supported formats
+    name: Check links and update table
+    runs-on: ubuntu-latest
+    env:
+      DOCS_PATH: ./docs/_build/html/
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - uses: actions/setup-python@v3
+        name: Install Python
+        with:
+          python-version: '3.10'
+
+      - name: Install build docs
+        shell: bash -l {0}
+        run: |
+          pip install .[docs]
+
+      - name: Update packages list
+        shell: bash -l {0}
+        run: |
+          # python generate_supported_format_table.py
+          echo "Table of Supported Format"
+          echo "========================="
+          cat ./docs/supported_formats/supported_formats.rst
+
+      - name: Check links
+        shell: bash -l {0}
+        run: |
+          cd docs
+          make linkcheck
+
+      - name: Build docs
+        shell: bash -l {0}
+        run: |
+          cd docs
+          make html
+
+      - uses: actions/upload-artifact@v3
+        with:
+          path: ${{ env.DOCS_PATH }}
+          name: docs_html
+
+  push_docs:
+    # This build is to push changes to gh-pages branch (https://hyperspy.org/rosettasciio)
+    needs: build_docs
+    name: Push to gh-pages
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name != 'pull_request' && github.repository == 'hyperspy/rosettasciio' }}
+    permissions:
+      # needs write permission to push the docs to gh-pages
+      contents: write
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+          ref: gh-pages
+
+      - uses: actions/download-artifact@v3
+        with:
+          name: docs_html
+          path: .
+
+      - name: list files
+        run: |
+          ls
+
+      - name: Git status
+        run: |
+          git status
+          git status -s
+          if [[ $(git status -s) ]]; then
+            HAS_CHANGES='true'
+          else
+            HAS_CHANGES='false'
+          fi
+          echo "HAS_CHANGES=${HAS_CHANGES}" >> $GITHUB_ENV
+
+      - name: Commit files
+        # Skip when there is nothing to commit
+        if: ${{ env.HAS_CHANGES == 'true' }}
+        run: |
+          git config --local user.email "github-actions[bot]@users.noreply.github.com"
+          git config --local user.name "github-actions[bot]"
+          git add -A
+          git commit -m "Update docs build" -a
+
+      - name: Push changes
+        if: ${{ env.HAS_CHANGES == 'true' }}
+        uses: ad-m/github-push-action@master
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          branch: gh-pages
diff --git a/CHANGES.rst b/CHANGES.rst
@@ -2,13 +2,13 @@ Changelog
 *********
 
 Changelog entries for the development version are available at
-https://rosettasciio.readthedocs.io/en/latest/user_guide/changes.html
+https://rosettasciio.readthedocs.io/en/latest/changes.html
 
 .. towncrier-draft-entries:: |release| [UNRELEASED]
 
 .. towncrier release notes start
 
-RosettaSciIO initiation
+Initiation (2022-07-23)
 =======================
 
 - RosettaSciIO was split out of the `HyperSpy repository 

diff --git a/COPYING.txt b/COPYING.txt
@@ -1,7 +1,7 @@
                     GNU GENERAL PUBLIC LICENSE
                        Version 3, 29 June 2007
 
- Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
  Everyone is permitted to copy and distribute verbatim copies
  of this license document, but changing it is not allowed.
 
@@ -645,7 +645,7 @@ the "copyright" line and a pointer to where the full notice is found.
     GNU General Public License for more details.
 
     You should have received a copy of the GNU General Public License
-    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 Also add information on how to contact you by electronic and paper mail.
 
@@ -664,11 +664,11 @@ might be different; for a GUI interface, you would use an "about box".
   You should also get your employer (if you work as a programmer) or school,
 if any, to sign a "copyright disclaimer" for the program, if necessary.
 For more information on this, and how to apply and follow the GNU GPL, see
-<http://www.gnu.org/licenses/>.
+<https://www.gnu.org/licenses/>.
 
   The GNU General Public License does not permit incorporating your program
 into proprietary programs.  If your program is a subroutine library, you
 may consider it more useful to permit linking proprietary applications with
 the library.  If this is what you want to do, use the GNU Lesser General
 Public License instead of this License.  But first, please read
-<http://www.gnu.org/philosophy/why-not-lgpl.html>.
+<https://www.gnu.org/philosophy/why-not-lgpl.html>.
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,6 +1,6 @@
 include *.txt
 include *.rst
 recursive-include rsciio *.pyx
-recursive-include doc *.txt *.rst
+recursive-include docs *.txt *.rst *.pdf *py *bat makefile
 include rsciio/tests/bruker_data/test_compilers.c
 
diff --git a/README.md b/README.md
@@ -4,7 +4,7 @@
 [![Documentation Status](https://readthedocs.org/projects/rosettasciio/badge/?version=latest)](https://rosettasciio.readthedocs.io/en/latest/?badge=latest)
 [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
 <!-- [![Python Version](https://img.shields.io/pypi/pyversions/rosettasciio.svg?style=flat)](https://pypi.python.org/pypi/rosettasciio) -->
-<!-- [![PyPi Version](http://img.shields.io/pypi/v/rosettasciio.svg?style=flat)](https://pypi.python.org/pypi/rosettasciio) -->
+<!-- [![PyPi Version](https://img.shields.io/pypi/v/rosettasciio.svg?style=flat)](https://pypi.python.org/pypi/rosettasciio) -->
 <!-- [![Anaconda Version](https://anaconda.org/conda-forge/rosettasciio/badges/version.svg)](https://anaconda.org/conda-forge/rosettasciio) -->
 <!-- [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.xxxxxxx.svg)](https://doi.org/10.5281/zenodo.xxxxxxx) -->
 

diff --git a/conda_environment.yml b/conda_environment.yml
@@ -0,0 +1,14 @@
+name: hyperspy-bundle
+channels:
+- conda-forge
+dependencies:
+- dask-core >=2.11
+- h5py
+- imageio
+- numba >=0.52
+- numpy
+- pint
+- python-box
+- pyyaml
+- scipy
+- sparse
diff --git a/docs/conf.py b/docs/conf.py
@@ -35,6 +35,8 @@
 
 intersphinx_mapping = {
     "hyperspy": ("https://hyperspy.org/hyperspy-doc/current/", None),
+    "pyusid": ("https://pycroscopy.github.io/pyUSID/", None),
+    "zarr": ("https://zarr.readthedocs.io/en/stable", None),
 }
 
 # Add any paths that contain templates here, relative to this directory.

diff --git a/docs/file_specification/edax/ImageIPR.pdf b/docs/file_specification/edax/ImageIPR.pdf
diff --git a/docs/file_specification/edax/SPECTRUM-V70.pdf b/docs/file_specification/edax/SPECTRUM-V70.pdf
diff --git a/docs/file_specification/edax/SpcMap-spd.file.format.pdf b/docs/file_specification/edax/SpcMap-spd.file.format.pdf
diff --git a/docs/file_specification/index.rst b/docs/file_specification/index.rst
@@ -0,0 +1,17 @@
+Reference File Specification
+============================
+
+This section contains reference documentation for file specification available
+publicly.
+
+.. _edax-file_specification:
+
+EDAX
+----
+
+The file specifications for the :ref:`edax <edax-format>` file formats made
+available publicly available from EDAX:
+
+- :download:`spc <edax/SPECTRUM-V70.pdf>`
+- :download:`spd <edax/SpcMap-spd.file.format.pdf>`
+- :download:`ipr <edax/ImageIPR.pdf>`
diff --git a/docs/index.rst b/docs/index.rst
@@ -41,5 +41,6 @@ Similarly, it takes a dictionary as input for file writers.
 
    supported_formats/index
    interoperatibility
+   file_specification/index
    changes
 
diff --git a/docs/supported_formats/blockfile.rst b/docs/supported_formats/blockfile.rst
@@ -9,7 +9,7 @@ electron diffraction (SPED) measurements, with a limited set of metadata. The
 header of the blockfile contains information about centering and distortions
 of the diffraction patterns, but is not applied to the signal during reading.
 Blockfiles only support data values of type
-`np.uint8 <http://docs.scipy.org/doc/numpy/user/basics.types.html>`_ (integers
+`np.uint8 <https://numpy.org/doc/stable/user/basics.types.html>`_ (integers
 in range 0-255).
 
 .. warning::
@@ -20,7 +20,7 @@ in range 0-255).
 
 Blockfiles are by default loaded in a "copy-on-write" manner using
 `numpy.memmap
-<http://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html>`_ .
+<https://numpy.org/doc/stable/reference/generated/numpy.memmap.html>`_ .
 For blockfiles ``load`` takes the ``mmap_mode`` keyword argument enabling
 loading the file using a different mode. However, note that lazy loading
 does not support in-place writing (i.e lazy loading and the "r+" mode

diff --git a/docs/supported_formats/edax.rst b/docs/supported_formats/edax.rst
@@ -19,12 +19,8 @@ Currently, loading an EDAX TEAM spectrum or spectrum image will load an
 open an issue in the `issues tracker <https://github.com/hyperspy/rosettasciio/issues>`_
 to alert the developers of the need.
 
-For further reference, file specifications for the formats are
-available publicly available from EDAX and are on Github
-(`spc <https://github.com/hyperspy/hyperspy/files/29506/SPECTRUM-V70.pdf>`_,
-`spd <https://github.com/hyperspy/hyperspy/files/29505/
-SpcMap-spd.file.format.pdf>`_, and
-`ipr <https://github.com/hyperspy/hyperspy/files/29507/ImageIPR.pdf>`_).
+For reference, :ref:`file specifications <edax-file_specification>`  for the edax
+file formats have been available publicly available from EDAX.
 
 Extra loading arguments for SPD file
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

diff --git a/docs/supported_formats/hspy.rst b/docs/supported_formats/hspy.rst
@@ -6,9 +6,9 @@ HSpy - HyperSpy's HDF5 Specification
 This is `HyperSpy's <https://hyperspy.org>`_ default format and it is the only one that guarantees that no
 information will be lost in the writing process and that supports saving data
 of arbitrary dimensions. It is based on the `HDF5 open standard
-<http://www.hdfgroup.org/HDF5/>`_. The HDF5 file format is supported by `many
+<https://www.hdfgroup.org/solutions/hdf5/>`_. The HDF5 file format is supported by `many
 applications
-<http://www.hdfgroup.org/products/hdf5_tools/SWSummarybyName.htm>`_.
+<https://support.hdfgroup.org/products/hdf5_tools/SWSummarybyName.htm>`_.
 Part of the specification is documented in :external+hyperspy:ref:`metadata_structure`.
 
 .. versionadded:: 1.2

diff --git a/docs/supported_formats/image.rst b/docs/supported_formats/image.rst
@@ -4,9 +4,9 @@ Image formats
 -------------
 
 RosettaSciIO can read and write data to `all the image formats
-<https://imageio.readthedocs.io/en/stable/formats.html>`_ supported by
-`imageio <https://imageio.readthedocs.io/>`_, which uses the 
-`Python Image Library (PIL/pillow) <https://pillow.readthedocs.io>`_.
+<https://imageio.readthedocs.io/en/stable/formats/index.html>`_ supported by
+`imageio <https://imageio.readthedocs.io/en/stable/>`_, which uses the 
+`Python Image Library (PIL/pillow) <https://pillow.readthedocs.io/en/stable/>`_.
 This includes ``.jpg``, ``.gif``, ``.png``, ``.pdf``, ``.tif``, etc.
 It is important to note that these image formats only support 8-bit files, and
 therefore have an insufficient dynamic range for most scientific applications.
@@ -41,7 +41,7 @@ Extra saving arguments
   :py:func:`~.matplotlib.pyplot.imshow`.
 - ``**kwds`` : keyword arguments supported by the individual file
   writers as documented at
-  https://imageio.readthedocs.io/en/stable/formats.html when exporting
+  `imageio <https://imageio.readthedocs.io/en/stable/formats/index.html>`__ when exporting
   an image without scalebar. When exporting with a scalebar, the keyword
   arguments are passed to the `pil_kwargs` dictionary of
   :py:func:`matplotlib.pyplot.savefig`

diff --git a/docs/supported_formats/mrcz.rst b/docs/supported_formats/mrcz.rst
@@ -4,7 +4,7 @@ MRCZ format
 -----------
 
 The ``.mrcz`` format is an extension of the CCP-EM MRC2014 file format.
-`CCP-EM MRC2014 <http://www.ccpem.ac.uk/mrc_format/mrc2014.php>`_ file format.
+`CCP-EM MRC2014 <https://www.ccpem.ac.uk/mrc_format/mrc2014.php>`_ file format.
 It uses the `blosc` meta-compression library to bitshuffle and compress files in
 a blocked, multi-threaded environment. The supported data types are:
 
@@ -15,9 +15,9 @@ It supports arbitrary meta-data, which is serialized into JSON.
 MRCZ also supports asynchronous reads and writes.
 
 Repository: https://github.com/em-MRCZ
-PyPI:       https://pypi.python.org/pypi/mrcz
+PyPI:       https://pypi.org/project/mrcz
 Citation:   Submitted.
-Preprint:   http://www.biorxiv.org/content/early/2017/03/13/116533
+Preprint:   https://www.biorxiv.org/content/10.1101/116533v1
 
 Support for this format is not enabled by default. In order to enable it
 install the `mrcz` and optionally the `blosc` Python packages.

diff --git a/docs/supported_formats/ripple.rst b/docs/supported_formats/ripple.rst
@@ -4,7 +4,7 @@ Ripple format
 -------------
 
 The ``.rpl`` format is an *open standard format* developed at NIST as native format for
-`Lispix <http://www.nist.gov/lispix/>`_ and is widely used to exchange
+`Lispix <https://www.nist.gov/services-resources/software/lispix>`_ and is widely used to exchange
 multidimensional data. However, it only supports data of up to three
 dimensions. It can also be used to exchange data with Bruker and used in
 combination with the :ref:`import-rpl` it is very useful for exporting data

diff --git a/docs/supported_formats/tiff.rst b/docs/supported_formats/tiff.rst
@@ -4,13 +4,13 @@ Tagges image file format (TIFF)
 -------------------------------
 
 RosettaSciIO can read and write 2D and 3D ``.tiff`` files using using
-Christoph Gohlke's `tifffile <https://github.com/cgohlke/tifffile/>`_ library.
+Christoph Gohlke's `tifffile <https://pypi.org/project/tifffile/>`__ library.
 In particular, it supports reading and
 writing of TIFF, BigTIFF, OME-TIFF, STK, LSM, NIH, and FluoView files. Most of
 these are uncompressed or losslessly compressed 2**(0 to 6) bit integer, 16, 32
 and 64-bit float, grayscale and RGB(A) images, which are commonly used in
 bio-scientific imaging. See `the library webpage
-<http://www.lfd.uci.edu/~gohlke/code/tifffile.py.html>`_ for more details.
+<https://pypi.org/project/tifffile/>`__ for more details.
 
 .. versionadded: 1.0
    Add support for writing/reading scale and unit to tif files to be read with

diff --git a/docs/supported_formats/usid.rst b/docs/supported_formats/usid.rst
@@ -95,12 +95,8 @@ signal for each named component in the dataset:
     Signal2D, title: green, dimensions: (|128, 128)>]
 
 h5USID files also support parameters or dimensions that have been varied non-uniformly.
-This capability is important in several spectroscopy techniques where the bias is varied as a
-`bi-polar triangular waveform <https://pycroscopy.github.io/pyUSID/auto_examples/beginner/plot_usi_dataset.html#values-for-each-dimension>`_
-rather than uniformly from the minimum value to the maximum value.
-So far, the reading of non-uniform axes is not implemented in RosettaSciIO and as the non-uniform information
-would be lost in the axes manager, the USID plugin will default to a warning
-when it encounters a parameter that has been varied non-uniformly:
+Currently, the reading of non-uniform axes is not implemented in RosettaSciIO, the USID plugin
+will default to a warning when it encounters a parameter that has been varied non-uniformly:
 
 .. code-block:: python
 
@@ -122,8 +118,7 @@ Writing
 
 Signals can be written to new h5USID files using the standard :py:meth:`~.signal.BaseSignal.save` function.
 Setting the ``overwrite`` keyword argument to ``True`` will append to the specified
-HDF5 file. All other keyword arguments will be passed to
-`pyUSID.hdf_utils.write_main_dataset() <https://pycroscopy.github.io/pyUSID/_autosummary/_autosummary/pyUSID.io.hdf_utils.html#pyUSID.io.hdf_utils.write_main_dataset>`_
+HDF5 file. All other keyword arguments will be passed to :py:func:`pyUSID.io.hdf_utils.model.write_main_dataset`.
 
 .. code-block:: python