Docstring improvement and check API links

.github/workflows/docs.yml

@@ -44,7 +44,7 @@ jobs:
       - name: Install build docs
         shell: bash -l {0}
         run: |
-          pip install .[docs]
+          pip install .'[all, docs]'
 
       - name: Update packages list
         shell: bash -l {0}
@@ -64,7 +64,7 @@ jobs:
         shell: bash -l {0}
         run: |
           cd docs
-          make html
+          make SPHINXOPTS="-W --keep-going" html
 
       - uses: actions/upload-artifact@v3
         with:

CONTRIBUTING.rst

@@ -2,8 +2,8 @@ Contributing
 ************
 
 RosettaSciIO is meant to be a community maintained project. We welcome contributions
-in the form of bug reports, documentation, code (in particular :ref:`new io plugins 
-<defining-plugins>`), feature requests, and more. In the following we refer to some 
+in the form of bug reports, documentation, code (in particular :ref:`new io plugins
+<defining-plugins>`), feature requests, and more. In the following we refer to some
 resources to help you make useful contributions.
 
 Issues
@@ -26,15 +26,15 @@ starting point. But don't hesitate also for significant code contributions, such
 as support for a new file format - if needed, we'll help you to get the code ready
 to common standards.
 
-Please refer to the 
+Please refer to the
 `HyperSpy developer guide <http://hyperspy.org/hyperspy-doc/current/dev_guide/intro.html>`_
 in order to get started and for detailed contributing guidelines.
 
 Lint
 ----
 To keep the code style consistent (and more readable), `black <https://black.readthedocs.io/>`_
 is used to check the code formatting. When the code doesn't comply with the expected formatting,
-the `lint <https://github.com/hyperspy/rosettasciio/actions/workflows/black.yml>`_ will fail. 
+the `lint <https://github.com/hyperspy/rosettasciio/actions/workflows/black.yml>`_ will fail.
 In practise, the code formatting can be fixed by installing ``black`` and running it on the
 source code or by using :ref:`pre-commit hooks <pre-commit-hooks>`.
 
@@ -59,7 +59,7 @@ To add or update test data:
       from rsciio.tests.registry_utils import update_registry
 
       update_registry()
-  
+
    On windows, you can use :ref:`pre-commit.ci <pre-commit-hooks>` by adding a message to
    the pull request to update the registry.
 
@@ -128,7 +128,7 @@ directory should contain the following files:
 
   * A function called ``file_reader`` with at least one attribute: ``filename``
     that returns the :ref:`standardized signal dictionary <interfacing-api>`.
-  * (optional) A function called ``file_writer`` with at least two attributes: 
+  * (optional) A function called ``file_writer`` with at least two attributes:
     ``filename`` and ``signal`` (a python dictionary) in that order.
 
 **Tests** covering the functionality of the plugin should be added to the
@@ -156,6 +156,12 @@ The *docstrings* are automatically added in the *user guide* using the following
     .. automodule:: rsciio.spamandeggs
        :members:
 
+The *docstrings* follow `Numpy docstring style <https://numpydoc.readthedocs.io>`_. The
+links to RosettaSciIO API and other Sphinx documented API are checked when building the documentation
+and broken links will raise warnings. In order to identify potentially broken links during pull
+request review, the `Docs <https://github.com/hyperspy/rosettasciio/actions/workflows/docs.yml>`_
+GitHub CI workflow is set to fail when the doc build raises warnings.
+
 .. Note ::
     It is advisable to clone the files of an existing plugin when initiating a new
     plugin.
@@ -164,6 +170,6 @@ The *docstrings* are automatically added in the *user guide* using the following
 Maintenance
 ===========
 
-Please refer to the 
+Please refer to the
 `HyperSpy developer guide <http://hyperspy.org/hyperspy-doc/current/dev_guide/intro.html>`_
 for maintenance guidelines.

docs/conf.py

@@ -14,6 +14,7 @@
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
 
+import numpydoc
 import pydata_sphinx_theme
 from packaging.version import Version
 
@@ -30,6 +31,10 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
+    # numpydoc is necessary to parse the docstring using sphinx
+    # otherwise the nitpicky option will raise many warnings
+    "numpydoc",
+    "sphinx_favicon",
     "sphinx.ext.autodoc",
     "sphinx.ext.githubpages",
     "sphinx.ext.intersphinx",
@@ -38,8 +43,15 @@
 ]
 
 intersphinx_mapping = {
+    "dask": ("https://docs.dask.org/en/latest", None),
     "hyperspy": ("https://hyperspy.org/hyperspy-doc/current/", None),
     "h5py": ("https://docs.h5py.org/en/stable/", None),
+    "matplotlib": ("https://matplotlib.org", None),
+    "mrcz": ("https://python-mrcz.readthedocs.io", None),
+    "numcodecs": ("https://numcodecs.readthedocs.io/en/stable", None),
+    "numpy": ("https://docs.scipy.org/doc/numpy", None),
+    "pooch": ("https://www.fatiando.org/pooch/latest", None),
+    "python": ("https://docs.python.org/3", None),
     "pyusid": ("https://pycroscopy.github.io/pyUSID/", None),
     "zarr": ("https://zarr.readthedocs.io/en/stable", None),
 }
@@ -78,15 +90,26 @@
         "image_light": "_static/logo_rec_oct22.svg",
         "image_dark": "_static/logo_rec_dark_oct22.svg",
     },
-    "favicons": [
-        {
-            "rel": "icon",
-            "href": "logo_sq.svg",
-        },
-    ],
     "header_links_before_dropdown": 6,
 }
 
+# -- Options for sphinx_favicon extension -----------------------------------
+
+favicons = {"rel": "icon", "href": "logo_sq.svg", "type": "image/svg+xml"}
+
+# Check links to API when building documentation
+nitpicky = True
+# Remove when fixed in hyperspy
+nitpick_ignore_regex = [(r"py:.*", r"hyperspy.api.*")]
+
+# -- Options for numpydoc extension -----------------------------------
+
+numpydoc_xref_param_type = True
+numpydoc_xref_ignore = {"type", "optional", "default", "of"}
+
+if Version(numpydoc.__version__) >= Version("1.6.0rc0"):
+    numpydoc_validation_checks = {"all", "ES01", "EX01", "GL02", "GL03", "SA01", "SS06"}
+
 # -- Options for towncrier_draft extension -----------------------------------
 
 # Options: draft/sphinx-version/sphinx-release

docs/supported_formats/supported_formats.rst

@@ -36,7 +36,7 @@
     +---------------------------------------------------------------------+-------------------------+--------+--------+--------+
     | :ref:`Images (imageio) <image-format>`                              | jpg, png, tif, ...      |    Yes |    Yes |    Yes |
     +---------------------------------------------------------------------+-------------------------+--------+--------+--------+
-    | :ref:`JEOL <jeol-format>`                                           | asw, map, img, pts, eds |    Yes |    No  |    No  |
+    | :ref:`JEOL <jeol-format>`                                           | asw, map, img, pts, eds |    Yes |    No  |    Yes |
     +---------------------------------------------------------------------+-------------------------+--------+--------+--------+
     | :ref:`MRC <mrc-format>`                                             | mrc                     |    Yes |    No  |    Yes |
     +---------------------------------------------------------------------+-------------------------+--------+--------+--------+

docs/supported_formats/zspy.rst

@@ -21,16 +21,16 @@ issues with the hdf5 format and is therefore suitable for saving
 
 
 When saving to `zspy <https://zarr.readthedocs.io/en/stable>`_, all supported objects in the signal's
-:py:attr:`~.signal.BaseSignal.metadata` is stored. This includes lists, tuples and signals.
+:py:attr:`hyperspy.api.signals.BaseSignal.metadata` is stored. This includes lists, tuples and signals.
 Please note that in order to increase saving efficiency and speed, if possible,
 the inner-most structures are converted to numpy arrays when saved. This
 procedure homogenizes any types of the objects inside, most notably casting
 numbers as strings if any other strings are present:
 
 By default, a :py:class:`zarr.storage.NestedDirectoryStore` is used, but other
 zarr store can be used by providing a :py:mod:`zarr.storage`
-instead as argument to the :py:meth:`~.signal.BaseSignal.save` or the
-:py:func:`~.io.load` function. If a ``.zspy`` file has been saved with a different
+instead as argument to the :py:meth:`hyperspy.api.signals.BaseSignal.save` or the
+:py:func:`hyperspy.api.load` function. If a ``.zspy`` file has been saved with a different
 store, it would need to be loaded by passing a store of the same type:
 
 .. code-block:: python

pyproject.toml

@@ -8,33 +8,33 @@ description = "Reading and writing scientific file formats"
 requires-python = ">=3.8"
 readme = "README.md"
 classifiers = [
-    "Development Status :: 4 - Beta",
-    "Intended Audience :: Developers",
-    "Intended Audience :: Science/Research",
-    "Topic :: Software Development :: Libraries",
-    "Topic :: Scientific/Engineering",
-    "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
-    "Programming Language :: Python :: 3.8",
-    "Programming Language :: Python :: 3.9",
-    "Programming Language :: Python :: 3.10",
-    "Programming Language :: Python :: 3.11",
-    "Operating System :: Microsoft :: Windows",
-    "Operating System :: POSIX",
-    "Operating System :: Unix",
-    "Operating System :: MacOS",
+  "Development Status :: 4 - Beta",
+  "Intended Audience :: Developers",
+  "Intended Audience :: Science/Research",
+  "Topic :: Software Development :: Libraries",
+  "Topic :: Scientific/Engineering",
+  "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
+  "Programming Language :: Python :: 3.8",
+  "Programming Language :: Python :: 3.9",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Operating System :: Microsoft :: Windows",
+  "Operating System :: POSIX",
+  "Operating System :: Unix",
+  "Operating System :: MacOS",
 ]
 dependencies = [
-    "dask[array]>=2021.3.1",
-    "python-dateutil",
-    "h5py>=2.3",
-    "imageio>=2.16",
-    "numba>=0.52",
-    "numpy>=1.20.0",
-    "pint>=0.8",
-    "python-box>=6,<7",
-    "pyyaml",
-    "scipy>=1.5.0",
-    "sparse",
+  "dask[array]>=2021.3.1",
+  "python-dateutil",
+  "h5py>=2.3",
+  "imageio>=2.16",
+  "numba>=0.52",
+  "numpy>=1.20.0",
+  "pint>=0.8",
+  "python-box>=6,<7",
+  "pyyaml",
+  "scipy>=1.5.0",
+  "sparse",
 ]
 dynamic = ["version"]
 
@@ -53,21 +53,35 @@ scalebar_export = ["matplotlib-scalebar", "matplotlib>=3.1.3"]
 tiff = ["tifffile>=2020.2.16", "imagecodecs>=2020.1.31"]
 usid = ["pyUSID"]
 zspy = ["zarr"]
-tests = ["pooch", "pytest>=3.6", "pytest-xdist", "pytest-rerunfailures", "pytest-cov"]
-docs = ["pydata-sphinx-theme>=0.13", "sphinxcontrib-towncrier", "towncrier"]
+tests = [
+  "pooch",
+  "pytest>=3.6",
+  "pytest-xdist",
+  "pytest-rerunfailures",
+  "pytest-cov"
+]
+docs = [
+  "numpydoc",
+  "pooch",
+  "pydata-sphinx-theme>=0.13",
+  "sphinx",
+  "sphinx-favicon",
+  "sphinxcontrib-towncrier",
+  "towncrier"
+]
 all = [
-    "rosettasciio[blockfile]",
-    "rosettasciio[mrcz]",
-    "rosettasciio[scalebar_export]",
-    "rosettasciio[tiff]",
-    "rosettasciio[usid]",
-    "rosettasciio[zspy]",
+  "rosettasciio[blockfile]",
+  "rosettasciio[mrcz]",
+  "rosettasciio[scalebar_export]",
+  "rosettasciio[tiff]",
+  "rosettasciio[usid]",
+  "rosettasciio[zspy]",
 ]
 dev = [
-    "black",
-    "rosettasciio[docs]",
-    "rosettasciio[all]",
-    "rosettasciio[tests]"
+  "black",
+  "rosettasciio[docs]",
+  "rosettasciio[all]",
+  "rosettasciio[tests]"
 ]
 
 [tool.pytest.ini_options]

rsciio/_docstrings.py

@@ -20,6 +20,7 @@
         Filename of the file to read or corresponding `pathlib.Path`.
     """
 
+
 SIGNAL_DOC = """signal : dict
         Dictionary containing the signal object.
         Should contain the following fields:
@@ -32,33 +33,80 @@
         - 'metadata' – dictionary containing the metadata tree
     """
 
-LAZY_DOC = """lazy : bool, Default=False
+
+LAZY_DOC = """lazy : bool, default=False
         Whether to open the file lazily or not.
     """
 
-ENCODING_DOC = """encoding : str, Default="latin-1"
+
+LAZY_UNSUPPORTED_DOC = """lazy : bool, default=False
+        Lazy loading is not supported.
+    """
+
+
+CHUNKS_DOC = """chunks : tuple of int or None, default=None
+        Define the chunking used for saving the dataset. If ``None``, calculates
+        chunks for the signal, with preferably at least one chunk per signal
+        space.
+    """
+
+
+SHOW_PROGRESSBAR_DOC = """show_progressbar : bool, default=True
+        Whether to show the progressbar or not.
+    """
+
+
+ENCODING_DOC = """encoding : str, default="latin-1"
         The encoding used to read the content of the file. Different file
         encodings, such as ``"utf8"`` can be set via this argument.
     """
 
-ENDIANESS_DOC = """endianess : str, Default="<"
-        ``"<"`` or ``">"``, depending on how the bits are written to 
+
+ENDIANESS_DOC = """endianess : str, default="<"
+        ``"<"`` or ``">"``, depending on how the bits are written to
         the file.
     """
 
-MMAP_DOC = """mmap_mode : {None, "r+", "r", "w+", "c"}, Default=None
-        Argument passed to :py:func:`numpy.memmap`. If None (default), the
-        value is ``"r"`` when ``lazy=True``, otherwise it is ``"c"``.
+
+MMAP_DOC = """mmap_mode : {None, "r+", "r", "w+", "c"}, default=None
+        Argument passed to :py:class:`numpy.memmap`. A memory-mapped array is
+        stored on disk, and not directly loaded into memory.  However, it can be
+        accessed and sliced like any ndarray.  Lazy loading does not support in-place writing (i.e lazy loading
+        and the ``"r+"`` mode are incompatible).
+        If ``None`` (default), the value is ``"r"`` when ``lazy=True``, otherwise
+        it is ``"c"``.
+    """
+
+
+COMPRESSION_HDF5_DOC = """compression : None, 'gzip', 'szip', 'lzf', default='gzip'
+        Compression can significantly increase the saving speed. If file size is not
+        an issue, it can be disabled by setting ``compression=None``.
+        RosettaSciIO uses h5py for reading and writing HDF5 files and, therefore,
+        it supports all `compression filters supported by h5py
+        <https://docs.h5py.org/en/stable/high/dataset.html#dataset-compression>`_.
+        The default is ``'gzip'``. Also see notes below.
+    """
+
+
+COMPRESSION_HDF5_NOTES_DOC = """It is possible to enable other compression filters such as ``blosc`` by
+    installing e.g. `hdf5plugin <https://github.com/silx-kit/hdf5plugin>`_.
+    Similarly, the availability of ``'szip'`` depends on the HDF5 installation.
+    If not available an error will be raised. Be aware that loading those
+    files will require installing the package providing the compression filter
+    and it may thus not be possible to load it on some platforms.
+    Only ``compression=None`` and ``compression='gzip'`` are available on all
+    platforms. For more details, see the `h5py documentation
+    <https://docs.h5py.org/en/stable/faq.html#what-compression-processing-filters-are-supported>`_.
     """
 
 
 RETURNS_DOC = """Returns
     -------
 
-    list of dicts
+    list of dict
         List of dictionaries containing the following fields:
 
-        - 'data' – multidimensional numpy array
+        - 'data' – multidimensional :py:class:`numpy.ndarray` or :py:class:`dask.array.Array`
         - 'axes' – list of dictionaries describing the axes
           containing the fields 'name', 'units', 'index_in_array', and
           either 'size', 'offset', and 'scale' or a numpy array 'axis'