doc: add sphinx

.github/workflows/doc.yml

@@ -0,0 +1,48 @@
+name: Build-sphinx-doc
+
+on:
+  push:
+    # branches: [main]
+  pull_request:
+    # branches: [main]
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e .[doc]
+          pip install ghp-import
+
+      - name: Format check for jupyter notebooks
+        run: |
+          black --check doc
+
+      - name: Build HTML
+        env:
+          TOKEN: ${{ secrets.TOKEN }}
+        run: |
+          sed -i '/nb_execution_mode = "off"/d' doc/source/conf.py # enable executing jupyter notebook
+          cd doc
+          make html
+
+      - name: Remove ampersand encoding and token in the generated html
+        run: |
+          # See issues about ampersand encoding at https://github.com/executablebooks/MyST-Parser/issues/760
+          sed -i 's/amp;//g' doc/build/html/Tutorial/Oceans_3.0_API_Tutorial.html
+          sed -i 's/token=[a-zA-Z0-9-]\+/token=YOUR_TOKEN/g' doc/build/html/Tutorial/onc_Library_Tutorial.html
+
+      - name: Run ghp-import to generate GitHub Pages
+        run: |
+          ghp-import -npfo doc/build/html

CONTRIBUTING.md

@@ -90,7 +90,7 @@ $ isort src tests
 Or use tox
 
 ```shell
-$ tox -e format
+$ tox -e format # This also formats doc folder
 ```
 
 ### Linter
@@ -121,7 +121,68 @@ $ tox -e lint -- --fix src tests
 
 ## Set up a development environment for documentation
 
-WIP.
+[Sphinx](https://www.sphinx-doc.org) is used for documentation, with several extensions.
+
+- [MyST](https://myst-parser.readthedocs.io): Parse markdown files.
+- [MyST_NB](https://myst-nb.readthedocs.io): Compile jupyter notebooks.
+- [AutoAPI](https://sphinx-autoapi.readthedocs.io/): Auto generate API documentation.
+- [napoleon](https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html): Support NumPy style docstrings.
+- [viewcode](https://www.sphinx-doc.org/en/master/usage/extensions/viewcode.html): Add links to source code.
+- [copybutton](https://sphinx-copybutton.readthedocs.io/): Add a copy button for code blocks.
+
+### Creating a virtual environment for documentation
+
+Follow the same step in "Set up a development environment", and install dependencies for doc instead of dev.
+
+```shell
+# Install onc library and doc dependencies in editable mode
+$ pip install -e .[doc]
+```
+
+### Building doc
+
+```shell
+$ cd doc
+$ make html
+# For Windows, use ".\make html" or "doc\make html" without cd
+```
+
+Sometimes you need to clean the build folder to force a rebuild.
+
+```shell
+$ make clean
+# For Windows, use ".\make clean" or "doc\make html" without cd
+```
+
+There is a Sphinx extension called [sphinx-autobuild](https://github.com/executablebooks/sphinx-autobuild) to watch a Sphinx directory and rebuild the documentation when a change is detected. It also includes a livereload enabled web server.
+
+```shell
+$ sphinx-autobuild doc/source doc/build/html --watch "*.md" --watch src
+```
+
+Or use tox
+
+```shell
+tox -e doc-autobuild
+```
+
+### Jupyter notebooks
+
+Jupyter notebooks are formatted using [black](https://black.readthedocs.io/en/stable/).
+
+```shell
+$ black doc
+```
+
+Or use tox
+
+```shell
+$ tox -e format # This also formats src and tests folder
+```
+
+[MyST_NB](https://myst-nb.readthedocs.io) supports executing jupyter notebooks when building the documentation. This is turned off in `conf.py` for faster local development, and enabled in GitHub Actions. Currently only `onc_Library_Tutorial.ipynb` will be run during the build.
+
+For a better git diff, all the outputs are cleared in the jupyter notebooks.
 
 ## Make a pull request
 
@@ -172,6 +233,8 @@ $ tox
 
 3. If you are working on a fork, add an Actions secret called TOKEN, and manually trigger the the "Formatting, Linting and Testing" workflow for your feature branch after pushing your commits.
 
+4. If you make changes to any jupyter notebooks, make sure to clear all the outputs and run `$ black doc`.
+
 ### After making the pull request
 
 If you find that some GitHub checks failed, you can check the log in the Actions tab to see what went wrong.

doc/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

doc/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

doc/source/API_Guide.md

@@ -0,0 +1,119 @@
+# API Guide
+
+## Oceans 3.0 API overview
+
+Check [here](https://wiki.oceannetworks.ca/display/O2A/API+Overview) for more information.
+
+## Glossary of terms
+
+Check [here](https://wiki.oceannetworks.ca/display/O2A/Glossary+of+Terms) for more information.
+
+## The ONC class
+
+The [ONC](#onc.onc.ONC) class provides a wrapper for Oceans 3.0 API requests. All the client library's functionality is provided as methods of this class.
+
+Create an ONC object to access this library's functionalities.
+
+## Discovery methods
+
+Discovery methods can be used to search for available locations, deployments, device categories, devices, properties, and data products. They support numerous filters and might resemble an "advanced search" function for ONC data sources.
+
+Use discovery methods to:
+
+- Obtain the identification codes required to use other API services.
+- Explore what's available in a certain location or device.
+- Obtain the deployment dates for a device.
+- List available data products for download in a particular device or location.
+
+::: {note}
+
+- Locations can contain other locations.
+  - "Cambridge bay" may contain separate children locations for its underwater network and shore station.
+- Locations can contain device categories, which contain devices, which contain properties.
+- Searches can be performed without considering the hierarchy mentioned above.
+  - You can search for locations with data on a specific property or search for all properties in a specific location.
+
+:::
+
+|                          Method                           |                       Description                        |                                   API Endpoint                                   |
+| :-------------------------------------------------------: | :------------------------------------------------------: | :------------------------------------------------------------------------------: |
+|         [getLocations](#onc.onc.ONC.getLocations)         |           Returns a filtered list of locations           |        [/locations](https://data.oceannetworks.ca/OpenAPI#get-/locations)        |
+| [getLocationHierarchy](#onc.onc.ONC.getLocationHierarchy) | Returns a filtered tree of locations with their children |   [/locations/tree](https://data.oceannetworks.ca/OpenAPI#get-/locations/tree)   |
+|       [getDeployments](#onc.onc.ONC.getDeployments)       |          Returns a filtered list of deployments          |      [/deployments](https://data.oceannetworks.ca/OpenAPI#get-/deployments)      |
+|  [getDeviceCategories](#onc.onc.ONC.getDeviceCategories)  |       Returns a filtered list of device categories       | [/deviceCategories](https://data.oceannetworks.ca/OpenAPI#get-/deviceCategories) |
+|           [getDevices](#onc.onc.ONC.getDevices)           |            Returns a filtered list of devices            |          [/devices](https://data.oceannetworks.ca/OpenAPI#get-/devices)          |
+|        [getProperties](#onc.onc.ONC.getProperties)        |          Returns a filtered list of properties           |       [/properties](https://data.oceannetworks.ca/OpenAPI#get-/properties)       |
+|      [getDataProducts](#onc.onc.ONC.getDataProducts)      |         Returns a filtered list of data products         |     [/dataProducts](https://data.oceannetworks.ca/OpenAPI#get-/dataProducts)     |
+
+## Data product download methods
+
+Data product download methods allow you to request and download more than 120 different types of [ONC data products](https://wiki.oceannetworks.ca/display/O2A/Available+Data+Products), with granular control over what data to obtain, from where, and in what time frame. They are comparable to the download functionality from ONC's [Data Search](https://data.oceannetworks.ca/DataSearch) tool.
+
+Examples of usage include:
+
+- Downloading PNG plots of sensor readings in a device
+- Downloading sensor readings as .mat files, text files or in commercial manufacturer formats
+- Downloading compressed or raw audio files from hydrophones
+
+:::{note}
+
+If the data product requested doesn't exist in our archive, it will be generated by our servers before your download starts.
+
+:::
+
+|                         Method                          |                  Description                  |                                               API Endpoint                                               |
+| :-----------------------------------------------------: | :-------------------------------------------: | :------------------------------------------------------------------------------------------------------: |
+|    [orderDataProduct](#onc.onc.ONC.orderDataProduct)    | Request, run and download <br> a data product |                                                                                                          |
+|  [requestDataProduct](#onc.onc.ONC.requestDataProduct)  |            Request a data product             |  [/dataProductDelivery/request](https://data.oceannetworks.ca/OpenAPI#get-/dataProductDelivery/request)  |
+|      [runDataProduct](#onc.onc.ONC.runDataProduct)      |          Run a data product request           |      [/dataProductDelivery/run](https://data.oceannetworks.ca/OpenAPI#get-/dataProductDelivery/run)      |
+| [downloadDataProduct](#onc.onc.ONC.downloadDataProduct) |      Download an available data product       | [/dataProductDelivery/download](https://data.oceannetworks.ca/OpenAPI#get-/dataProductDelivery/download) |
+
+## Near real-time data access methods
+
+Near real-time (as fast as they get into our database) data access methods allow the extraction of sensor data as time-series, either as processed scalar data with [Quality Assurance and Control flags](https://wiki.oceannetworks.ca/display/DP/Quality+Assurance+Quality+Control) (QAQC) or directly as raw data obtained from the device in its specific output format. In contrast to the Data product download methods, this data can be downloaded directly without waiting for any kind of generation process.
+
+Common use cases include:
+
+- Plotting time-series from properties in a specific time frame or in "near real-time"
+- Quickly obtaining the latest reading from a particular sensor
+- Obtaining raw unprocessed data from our instruments (data might require processing to be readable)
+
+::: {note}
+
+The methods `getDirectByLocation()` and `getDirectRawByLocation()` obtain data readings from a location no matter what device it came from (hence the need to specify a _device category code_ instead of a single _device code_). You might want to obtain data by location instead of by device, as individual devices are often replaced/repositioned.
+
+Each request to our API can return a maximum of 100,000 samples; larger data requests must be downloaded as a sequence of pages. Use the _allPages_ parameter to automatically download all pages required for your requested time frame.
+
+:::
+
+|                            Method                             |                              Description                              |                                      API Endpoint                                      |
+| :-----------------------------------------------------------: | :-------------------------------------------------------------------: | :------------------------------------------------------------------------------------: |
+|    [getDirectByLocation](#onc.onc.ONC.getDirectByLocation)    | Obtain scalar data readings <br> from a device category in a location | [/scalardata/location](https://data.oceannetworks.ca/OpenAPI#get-/scalardata/location) |
+|      [getDirectByDevice](#onc.onc.ONC.getDirectByDevice)      |               Obtain scalar data readings from a device               |   [/scalardata/device](https://data.oceannetworks.ca/OpenAPI#get-/scalardata/device)   |
+| [getDirectRawByLocation](#onc.onc.ONC.getDirectRawByLocation) |  Obtain raw data readings <br> from a device category in a location   |    [/rawdata/location](https://data.oceannetworks.ca/OpenAPI#get-/rawdata/location)    |
+|   [getDirectRawByDevice](#onc.onc.ONC.getDirectRawByDevice)   |                Obtain raw data readings from a device                 |      [/rawdata/device](https://data.oceannetworks.ca/OpenAPI#get-/rawdata/device)      |
+
+## Archive file download methods
+
+These methods allow users to directly download previously generated data product files from our archive.
+
+ONC systems auto-generate and archive files of different types at set time intervals. These archived files can be downloaded without waiting for a generation process to finish (potentially faster than [Data product download methods](#data-product-download-methods)).
+
+::: {note}
+
+Archived files have a unique filename (e.g. "NAXYS_HYD_007_20091231T235919.476Z-spect.png") that includes the device code ("NAXYS_HYD_007") and the UTC date-time when the data in the file started being measured ("20091231T235919.476Z"). The filename might contain additional information.
+
+:::
+
+::: {caution}
+
+Due to security regulations, some very recent files (e.g. hydrophone .wav files in the last hour) might not be made immediately available.
+
+:::
+
+|                       Method                        |                              Description                               |                                       API Endpoint                                       |
+| :-------------------------------------------------: | :--------------------------------------------------------------------: | :--------------------------------------------------------------------------------------: |
+| [getListByLocation](#onc.onc.ONC.getListByLocation) | Get a list of archived files <br> for a device category in a location  | [/archivefile/location](https://data.oceannetworks.ca/OpenAPI#get-/archivefile/location) |
+|   [getListByDevice](#onc.onc.ONC.getListByDevice)   |               Get a list of archived files for a device                |   [/archivefile/device](https://data.oceannetworks.ca/OpenAPI#get-/archivefile/device)   |
+|           [getFile](#onc.onc.ONC.getFile)           |                Download a file with the given filename                 | [/archivefile/download](https://data.oceannetworks.ca/OpenAPI#get-/archivefile/download) |
+|    [getDirectFiles](#onc.onc.ONC.getDirectFiles)    | Download a list of archived files <br> that match the filters provided |                                                                                          |