Uses conda to load all documentation deps

.readthedocs.yaml

@@ -18,7 +18,6 @@ conda:
 
 python:
   install:
-    - requirements: docs/requirements.txt
     - requirements: requirements.txt
     - method: pip
       path: .

docs/environment.yaml

@@ -1,4 +1,4 @@
-name: readthedocs
+name: '2020_04_30'
 channels:
   - conda-forge
   - defaults
@@ -27,5 +27,34 @@ dependencies:
   - wheel=0.34.2=py_1
   - xz=5.2.5=h516909a_0
   - zlib=1.2.11=h516909a_1006
-prefix: /home/jsybran/anaconda3/envs/readthedocs
+  - pip:
+    - alabaster==0.7.12
+    - babel==2.8.0
+    - chardet==3.0.4
+    - commonmark==0.9.1
+    - docutils==0.16
+    - idna==2.9
+    - imagesize==1.2.0
+    - jinja2==2.11.2
+    - markupsafe==1.1.1
+    - packaging==20.3
+    - pbr==5.4.5
+    - pygments==2.6.1
+    - pyparsing==2.4.7
+    - pytz==2020.1
+    - recommonmark==0.6.0
+    - requests==2.23.0
+    - snowballstemmer==2.0.0
+    - sphinx==3.0.3
+    - sphinx-autodoc-typehints==1.10.3
+    - sphinx-rtd-theme==0.4.3
+    - sphinxcontrib-apidoc==0.3.0
+    - sphinxcontrib-applehelp==1.0.2
+    - sphinxcontrib-devhelp==1.0.2
+    - sphinxcontrib-htmlhelp==1.0.3
+    - sphinxcontrib-jsmath==1.0.1
+    - sphinxcontrib-qthelp==1.0.3
+    - sphinxcontrib-serializinghtml==1.1.4
+    - urllib3==1.25.9
+prefix: /home/jsybran/anaconda3/envs/2020_04_30
 

docs/help/write_docs.md

@@ -98,10 +98,56 @@ push a commit to master. Behind the scenes this service will build our api
 documentation read in all of our `.rst` and `.md` files for us. This process
 will take a while, but the result should appear online after a few minutes.
 
+### Updating Dependencies for Read the Docs
+
+The hardest part about ReadTheDocs is getting the remote server to properly
+install all dependencies needed to install the docs and build agatha. If the
+build fails remotely, but works locally (by running `make html` in the `docs`
+directory) then this is almost certainly the issue. Take a look at
+`.readthedocs.yaml` to see how dependencies are loaded.
+
+Pretty much, there are different types of dependencies that are specified in
+various confutation and requirements files. As always 'requirements.txt' in the
+root directory contains all the modules needed to run Agatha. However, some data
+dependencies are large, and we don't actually need them to build the docs. For
+example, spaCy models and BERT transformers. These now live in
+`data_requirements.txt` and will not be loaded by the remote document server.
+
+Lastly, there's a bunch of dependencies that aren't necessary for Agatha
+generally, but are necessary if you're going to build the docs. For instance,
+the particular version of [Sphinx][1] or the couple of extensions we use. These
+dependencies. We've had some issues making sure these dependencies are loaded
+properly through requirements files, so we turn to [conda].
+
+ReadTheDocs loads conda first and loads whatever we specify in our
+`docs/environment.yaml` file. Take a look at [these docs][9] for how conda loads
+from yaml files. Conda also allows us to specify non-python dependencies, such
+as [protobuf][10] that is necessary to build agatha.
+
+To update the conda environment used by ReadTheDocs, you need to [create a conda
+environment yaml file][8]. However, as developers, we often will just want a
+`requirements.txt` file to add to an already-existing environment. Here's the
+best way to add a documentation dependency, or a dependency that cannot be
+installed via `pip`.
+
+ 1. Run `conda env create -n <unique_name> -f docs/environment.yaml` to create a
+    new environment with all of the previously nessesary requirements.
+ 2. If the new dependency can be installed via pip, add it to the
+    `docs/requirements.txt` file. Then run `pip install -r docs/requirements.txt`.
+ 3. If the new dependency is only available through conda, simply run `conda
+    install <new_dep>`
+ 4. Run `conda env export > docs/environment.yaml` to update the stored conda
+    environment.
+ 5. Add all changed files to a commit and push it online. Make sure the
+    ReadTheDocs build succeeds.
 
 [1]:https://www.sphinx-doc.org/en/master/index.html
 [2]:https://readthedocs.org/
 [3]:https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html
 [4]:https://docs.python.org/3/library/typing.html
 [5]:https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
 [6]:https://daringfireball.net/projects/markdown/syntax
+[7]:https://www.anaconda.com/products/individual
+[8]:https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#sharing-an-environment
+[9]:https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-from-an-environment-yml-file
+[10]:https://developers.google.com/protocol-buffers

docs/requirements.txt

@@ -1,3 +1,10 @@
+# WARNING: This file will NOT be loaded by ReadTheDocs.org. Instead, this file
+# exists as an easy way for developers to add the documentation dependencies to
+# an existing environment.
+
+# Read help/write_docs.md for more information about how to update the
+# ReadTheDocs dependencies. But tl;dr: update environment.yaml
+
 # Generates Documentation
 sphinx>=3.0.0