Minor fixes to docs, and docs builds added to CI (#319)

* Minor doc fixes, including resolving broken links and fixing missing embedded images (#313).

* Added docs html and pdf builds to CI, with readthedocs docker image used to replicate their build environment. 




Co-authored-by: Janis Klaise <jk@seldon.io>

.github/workflows/ci.yml

@@ -50,21 +50,24 @@ jobs:
 
   docs:
 
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-18.04
+
+    container:
+      image: readthedocs/build:latest
+      options: --user root
 
     steps:
       - uses: actions/checkout@v2
-      - name: Set up Python 3.x
-        uses: actions/setup-python@v2
-        with:
-          python-version: '3.8'
       - name: Install dependencies
         run: |
-          sudo apt-get install pandoc
-          pip install -r requirements/docs.txt
-          pip freeze
-      - name: Build documentation
+          pip3 install -r requirements/docs.txt
+          pip3 freeze
+      - name: Build documentation to html
+        run: |
+          make build_docs
+      - name: Build documentation to pdf via latex
         run: |
+          make build_latex
 
   licenses:
 

.readthedocs.yml

@@ -5,6 +5,11 @@
 # Required
 version: 2
 
+build:
+  image: latest # Python 3.8 available on latest
+  apt_packages:
+    - pandoc
+
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
    configuration: doc/source/conf.py
@@ -17,4 +22,4 @@ formats:
 python:
    version: 3.8
    install:
-   - requirements: requirements/docs.txt
+   - requirements: requirements/docs.txt

Makefile

@@ -18,6 +18,10 @@ mypy: ## Run typeckecking according to mypy configuration in setup.cfg
 build_docs: ## Build the documentation
 	$(MAKE) -C doc html
 
+.PHONY: build_latex
+build_latex: ## Build the documentation into a pdf
+	$(MAKE) -C doc latexpdf
+
 .PHONY: clean_docs
 clean_docs: ## Clean the documentation build
 	$(MAKE) -C doc clean

alibi_detect/models/tensorflow/embedding.py

@@ -49,6 +49,7 @@ def __init__(
             last_hidden_state, hidden_state or hidden_state_cls.
 
             From the HuggingFace documentation:
+
             - pooler_output
                 Last layer hidden-state of the first token of the sequence
                 (classification token) further processed by a Linear layer and a Tanh

alibi_detect/models/tensorflow/pixelcnn.py

@@ -173,11 +173,15 @@ def __init__(self,
                  name='shift'):
         """Instantiates the `Shift` bijector which computes `Y = g(X; shift) = X + shift`
         where `shift` is a numeric `Tensor`.
-        Args:
-          shift: Floating-point `Tensor`.
-          validate_args: Python `bool` indicating whether arguments should be
-            checked for correctness.
-          name: Python `str` name given to ops managed by this object.
+
+        Parameters
+        ----------
+        shift
+            Floating-point `Tensor`.
+        validate_args
+            Python `bool` indicating whether arguments should be checked for correctness.
+        name
+            Python `str` name given to ops managed by this object.
         """
         with tf.name_scope(name) as name:
             dtype = dtype_util.common_dtype([shift], dtype_hint=tf.float32)

doc/source/conf.py

@@ -89,7 +89,8 @@
     "fbprophet",
     "torch",
     "transformers",
-    "tqdm"
+    "tqdm",
+    "dill"
 ]
 
 # Napoleon settings
@@ -175,16 +176,24 @@
 
 latex_elements = {
     # The paper size ('letterpaper' or 'a4paper').
-    #
     # 'papersize': 'letterpaper',
-    # The font size ('10pt', '11pt' or '12pt').
     #
+    # The font size ('10pt', '11pt' or '12pt').
     # 'pointsize': '10pt',
+    #
     # Additional stuff for the LaTeX preamble.
+    # - Replace unicode characters with utf8. 
+    #   (U+2588 and U+258E are used in tqdm progress bars)
+    # - Use enumitem for lists to prevent "too deeply nested" latex error
+    'preamble': r''' 
+        \DeclareUnicodeCharacter{2588}{=}
+        \DeclareUnicodeCharacter{258E}{|} 
+
+        \usepackage{enumitem}
+        \setlistdepth{99}
+        ''',
     #
-    # 'preamble': '',
     # Latex figure (float) alignment
-    #
     # 'figure_align': 'htbp',
 }
 

doc/source/examples/alibi_detect_deploy.nblink

@@ -1,3 +1,3 @@
 {
   "path": "../../../examples/alibi_detect_deploy.ipynb"
-}
+}

doc/source/landing.md

@@ -1,3 +1,5 @@
+:orphan:
+
 <p align="center">
   <img src="_static/Alibi_Detect_Logo.png" alt="Alibi Detect Logo" width="50%">
 </p>
@@ -6,4 +8,4 @@
 
 For more background on the importance of monitoring outliers and distributions in a production setting, check out [this talk](https://slideslive.com/38931758/monitoring-and-explainability-of-models-in-production?ref=speaker-37384-latest) from the *Challenges in Deploying and Monitoring Machine Learning Systems* ICML 2020 workshop, based on the paper [Monitoring and explainability of models in production](https://arxiv.org/abs/2007.06299) and referencing Alibi Detect.
 
-For a thorough introduction to drift detection, check out [Drift Detection: An Introduction with Seldon](https://youtu.be/JaPMFf0cKlM?t=1). The talk covers what drift is and why it pays to detect it, the different types of drift, how it can be detected in a principled manner and also describes the anatomy of a drift detector.
+For a thorough introduction to drift detection, check out [Drift Detection: An Introduction with Seldon](https://youtu.be/JaPMFf0cKlM?t=1). The talk covers what drift is and why it pays to detect it, the different types of drift, how it can be detected in a principled manner and also describes the anatomy of a drift detector.

doc/source/methods/seq2seq.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "[source](../api/alibi_detect.od.seq2eq.rst)"
+    "[source](../api/alibi_detect.od.seq2seq.rst)"
    ]
   },
   {

examples/cd_clf_cifar10.ipynb

@@ -420,7 +420,7 @@
    "source": [
     "## Detecting drift with a learned kernel\n",
     "\n",
-    "An alternative to training a classifier to output high probabilities for instances from the test window and low probabilities for instances from the reference window is to learn a kernel that outputs high similarities between instances from the same window and low similarities between instances from different windows. The kernel may then be used within an [MMD-test](../doc/source/examples/cd_mmd_cifar10.nblink) for drift. [Liu et al. (2020)](https://arxiv.org/abs/2002.09116) propose this learned approach and note that it is in fact a generalisation of the above classifier-based method. However, in this case we can train the kernel to directly optimise an estimate of the detector's power, which can result in superior performance.\n",
+    "An alternative to training a classifier to output high probabilities for instances from the test window and low probabilities for instances from the reference window is to learn a kernel that outputs high similarities between instances from the same window and low similarities between instances from different windows. The kernel may then be used within an [MMD-test](https://docs.seldon.io/projects/alibi-detect/en/latest/examples/cd_mmd_cifar10.html) for drift. [Liu et al. (2020)](https://arxiv.org/abs/2002.09116) propose this learned approach and note that it is in fact a generalisation of the above classifier-based method. However, in this case we can train the kernel to directly optimise an estimate of the detector's power, which can result in superior performance.\n",
     "\n",
     "### Detect drift with a learned PyTorch kernel\n",
     "\n",