Add nwb project analytics (#128)

.gitignore

@@ -156,4 +156,7 @@ cython_debug/
 # PyCharm
 .idea/
 
+# Custom data checkouts as part of the docs build
+docs/source/nwb-project-analytics/
+
 # End of https://www.toptal.com/developers/gitignore/api/python,jupyternotebooks

.readthedocs.yaml

@@ -0,0 +1,32 @@
+# .readthedocs.yml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+build:
+  os: ubuntu-20.04
+  tools:
+    python: '3.8'
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/source/conf.py
+
+# Build documentation with MkDocs
+#mkdocs:
+#  configuration: mkdocs.yml
+
+# Optionally build your docs in additional formats such as PDF and ePub
+formats: all
+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+  install:
+    - requirements: requirements.txt
+
+# Optionally include all submodules
+submodules:
+  include: all
+  recursive: true

README.rst

@@ -165,3 +165,9 @@ Once the mapping is defined, we can refer to specific types much like we would r
     }
 
 With this we can now easly link to elements in those packages. E.g., to links to the docs of ``pandas.DataFrame`` we would write ``:py:class:`~pandas.DataFrame``` in the docs. Similarly, to link to ``NWBFile`` in ``PyNWB`` we would write ``:py:class:`~pynwb.file.NWBFile``` in the docs. When including the ``~`` we tell Sphinx to ignore the package when rendering in the text, i.e., ``:py:class:`~pynwb.file.NWBFile``` (with ``~``) will render as ``NWBFile`` in the docs, whereas ``:py:class:`pynwb.file.NWBFile``` (without ``~``) will render as the full name ``pynwb.file.NWBFile``.
+
+
+NWB Software Analytics
+======================
+
+Software analytics are being managed via the :ref:`NWB Project Analytics <https://github.com/NeurodataWithoutBorders/nwb-project-analytics>`. This page checks out and renders the `nwb_project_analytics` automatically as part of the sphinx build process. The code statistics are being updated nightly via GitHub Actions on the `nwb_project_analytics` repository and are cached in the `data` folder of that repo. The rst sources and figures are then being generated automatically during the sphinx build.

docs/Makefile

@@ -14,6 +14,10 @@ help:
 
 .PHONY: help Makefile
 
+clean:
+	rm -rf source/nwb-project-analytics
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile

docs/source/conf.py

@@ -8,6 +8,7 @@
 # Import the definition of external links
 from conf_extlinks import extlinks
 from conf_extlinks import intersphinx_mapping
+from conf_nwb_analytics import build_project_analytics
 import sphinx_rtd_theme
 
 
@@ -54,7 +55,6 @@
 # html_logo = None
 html_logo = 'img/logo_brain_transp.png'
 
-
 html_theme_options = {
     # 'analytics_id': 'G-XXXXXXXXXX',  #  Provided by Google in your dashboard
     # 'analytics_anonymize_ip': False,
@@ -71,3 +71,6 @@
     'includehidden': True,
     'titles_only': False
 }
+
+# -- Build the nwb project analytics in the current directory
+build_project_analytics(target_dir=os.path.dirname(__file__))

docs/source/conf_extlinks.py

@@ -4,7 +4,7 @@
     'nwb_extension_git': ('https://github.com/nwb-extensions/%s', '%s'),
     'ndx-catalog': ('https://nwb-extensions.github.io/%s', '%s'),
     'pynwb-issue': ('https://github.com/NeurodataWithoutBorders/pynwb/issues/%s', 'pynwb#%s'),
-    'nwb2-biorxiv': ('https://www.biorxiv.org/content/10.1101/523035v1/%s',None),
+    'nwb2-biorxiv': ('https://www.biorxiv.org/content/10.1101/523035v1/%s', None),
     'nwb2-biorxiv-pdf': ('https://www.biorxiv.org/content/biorxiv/early/2019/01/17/523035.full.pdf%s', None),
     'allen-cell-type-atlas': ('http://celltypes.brain-map.org/%s', '%s'),
     'api-python': ('https://neurodatawithoutborders.github.io/api-python/build/html/api_usage.html%s', '%s'),
@@ -26,12 +26,12 @@
     'nwb-schema-language-docs': ('https://schema-language.readthedocs.io/en/latest/%s', '%s'),
     'nwb-storage-docs': ('https://nwb-storage.readthedocs.io/en/latest/%s', '%s'),
     'nwb-schema-docs': ('https://nwb-schema.readthedocs.io/en/latest/%s', '%s'),
-    'nwb-schema-release-notes':('http://nwb-schema.readthedocs.io/en/latest/format_release_notes.html%s', '%s'),
+    'nwb-schema-release-notes': ('http://nwb-schema.readthedocs.io/en/latest/format_release_notes.html%s', '%s'),
     'nwb-schema-src': ('https://github.com/NeurodataWithoutBorders/nwb-schema/%s', '%s'),
     'hdmf-common-schema-docs': ('https://hdmf-common-schema.readthedocs.io/en/stable/%s', '%s'),
     'hdmf-common-schema-src': ('https://github.com/hdmf-dev/hdmf-common-schema/%s', '%s'),
     'hdmf-z-docs': ('https://hdmf-zarr.readthedocs.io/en/latest/%s', '%s'),
-    'hdmf-z-src': ('https://github.com/hdmf-dev/hdmf-zarr/%s','%s'),
+    'hdmf-z-src': ('https://github.com/hdmf-dev/hdmf-zarr/%s', '%s'),
     'nwb-github-org': ('https://github.com/NeurodataWithoutBorders/%s', '%s'),
     'hdmf-github-org': ('https://github.com/hdmf-dev/%s', '%s'),
     'nwb-widgets-docs': ('https://github.com/NeurodataWithoutBorders/nwb-jupyter-widgets/%s', '%s'),
@@ -50,7 +50,8 @@
     'datajoint': ('https://www.datajoint.org/%s', '%s'),
     'datajoint-elements': ('https://www.datajoint.com/elements/%s', '%s'),
     'openscope-databook': ('https://alleninstitute.github.io/openscope_databook/%s', '%s'),
-    'openscope-project': ('https://alleninstitute.org/what-we-do/brain-science/research/mindscope-program/openscope/%s', '%s'),
+    'openscope-project': ('https://alleninstitute.org/what-we-do/brain-science/research/mindscope-program/openscope/%s',
+                          '%s'),
     'ibl-website': ('https://www.internationalbrainlab.com/%s', '%s'),
     'mindscope-program': ('https://alleninstitute.org/what-we-do/brain-science/research/mindscope-program/%s', '%s'),
     'jupyter-book': ('https://jupyterbook.org/en/stable/%s', '%s'),

docs/source/conf_nwb_analytics.py

@@ -0,0 +1,33 @@
+"""
+Configure and build the nwb project analytics sources
+"""
+import os
+
+
+def build_project_analytics(target_dir=None):
+    """
+    Clone the nwb-project-analytics repo and build the documentation
+
+    :param target_dir: Directory where to clone the repo or None if the repo should be
+                       cloned in the same directory as this file.
+    :return: Path where the generated rst sources are located
+    """
+    # Clone the project analytics repo
+    target_dir = os.path.join(os.path.dirname(__file__), "nwb-project-analytics")
+    if not os.path.exists(target_dir):
+        repo_url = "https://github.com/NeurodataWithoutBorders/nwb-project-analytics.git"
+        clone_command = f"git clone {repo_url} {target_dir}"
+        os.system(clone_command)
+        # remove the readme.rst file to avoid Sphinx warnings and possible unrelated linkcheck errors
+        os.remove(os.path.join(target_dir, "README.rst"))
+    else:
+        print(f"{target_dir} already checked out. Use 'make clean' to do a full rebuild.")
+
+    # Build the project analytics docs (assuming all dependencies are installed)
+    docs_dir = os.path.join(target_dir, "docs")
+    build_command = f"cd {docs_dir}; make html;"
+    os.system(build_command)
+
+    # Return the location of the rst sources
+    source_dir = os.path.join(docs_dir, "source")
+    return source_dir

docs/source/file_read/file_read.rst

@@ -8,7 +8,7 @@ This section provides an introduction on how to read NWB files in Python using P
 Reading With PyNWB
 ~~~~~~~~~~~~~~~~~~
 
-* See the PyNWB tutorial on :pynwb-docs:`Reading and Exploring an NWB File  <tutorials/general/read_basics.html>`
+* See the PyNWB tutorial on :pynwb-docs:`Reading and Exploring an NWB File  <tutorials/general/plot_read_basics.html>`
   for an introduction to how to read, explore, and do basic visualizations with an NWB file in Python.
 * The :pynwb-docs:`Query Intracellular Electrophysiology Metadata <tutorials/domain/plot_icephys_pandas.html>`
   PyNWB tutorial then focuses specifically on how to read and query metadata related to intracellular

docs/source/index.rst

@@ -41,6 +41,7 @@ for each of those tasks and point you to the best tools to use for your preferre
     :caption: For Developers
 
     nwb_source_codes
+    nwb-project-analytics/docs/source/index.rst
 
 
 .. Indices and tables

requirements.txt

@@ -1,5 +1,7 @@
 myst_parser
-sphinx
+sphinx>=4  # improved support for docutils>=0.17
 sphinx_rtd_theme>=1  # <1 does not work with docutils>=0.17
 sphinx_design
 sphinx-tabs
+sphinx-copybutton
+git+https://github.com/NeurodataWithoutBorders/nwb-project-analytics.git