Merge branch 'deployment-docs'

CartoDB · Mar 2, 2017 · 0b8ffc7 · 0b8ffc7
2 parents 273c925 + 89ccc5a
commit 0b8ffc7
Show file tree

Hide file tree

Showing 18 changed files with 429 additions and 10 deletions.
diff --git a/README.md b/README.md
@@ -28,7 +28,7 @@ Docker is available.
   [tasks/meta.py](https://github.com/CartoDB/bigmetadata/blob/master/tasks/meta.py#L76).
   There are six related tables, `obs_table`, `obs_column_table`, `obs_column`,
   `obs_column_tag`, `obs_tag`, and `obs_column_to_column`.  An overarching
-  denomralized view can be found in `obs_meta`.
+  denormalized view can be found in `obs_meta`.
 
 * __Catalog__: a [static HTML guide](https://cartodb.github.io/bigmetadata) to
   data in the observatory generated from the metadata.  Docs are generated

diff --git a/docs/source/development.rst b/docs/source/development.rst
@@ -64,12 +64,13 @@ over again.  These tasks are meant to take care of the most repetitive aspects.
    :members:
    :show-inheritance:
 
-Running pieces of the ETL
+Running and Re-Running Pieces of the ETL
 -------------------------
 
 When doing local development, it's advisable to run small pieces of the ETL
 locally to make sure everything works correctly.  You can use the ``make --
-run`` helper, documented in :ref:`run-any-task`.
+run`` helper, documented in :ref:`run-any-task`. There are several methods for
+re-running pieces of the ETL depending on the task and are described below:
 
 Using ``--force`` during development
 ************************************
@@ -80,6 +81,8 @@ overwriting all output it has already created.  For example, if you have
 a :ref:`tasks.util.TempTableTask` that you've modified in the course of
 development and need to re-run:
 
+.. code:: python
+
     from tasks.util import TempTableTask
     from tasks.meta import current_session
 

diff --git a/docs/source/tasks.au.data.rst b/docs/source/tasks.au.data.rst
@@ -0,0 +1,7 @@
+tasks.au.data module
+====================
+
+.. automodule:: tasks.au.data
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.au.geo.rst b/docs/source/tasks.au.geo.rst
@@ -0,0 +1,7 @@
+tasks.au.geo module
+===================
+
+.. automodule:: tasks.au.geo
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.au.rst b/docs/source/tasks.au.rst
@@ -0,0 +1,18 @@
+tasks.au package
+================
+
+Submodules
+----------
+
+.. toctree::
+
+   tasks.au.data
+   tasks.au.geo
+
+Module contents
+---------------
+
+.. automodule:: tasks.au
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.poi.rst b/docs/source/tasks.poi.rst
@@ -0,0 +1,7 @@
+tasks.poi module
+================
+
+.. automodule:: tasks.poi
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.rst b/docs/source/tasks.rst
@@ -6,6 +6,7 @@ Subpackages
 
 .. toctree::
 
+    tasks.au
     tasks.br
     tasks.ca
     tasks.es
@@ -23,7 +24,9 @@ Submodules
 
    tasks.carto
    tasks.meta
+   tasks.poi
    tasks.sched
+   tasks.socrata
    tasks.sphinx
    tasks.tags
    tasks.util

diff --git a/docs/source/tasks.socrata.rst b/docs/source/tasks.socrata.rst
@@ -0,0 +1,7 @@
+tasks.socrata module
+====================
+
+.. automodule:: tasks.socrata
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.ny.nyc.acris.rst b/docs/source/tasks.us.ny.nyc.acris.rst
@@ -0,0 +1,7 @@
+tasks.us.ny.nyc.acris module
+============================
+
+.. automodule:: tasks.us.ny.nyc.acris
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.ny.nyc.columns.rst b/docs/source/tasks.us.ny.nyc.columns.rst
@@ -0,0 +1,7 @@
+tasks.us.ny.nyc.columns module
+==============================
+
+.. automodule:: tasks.us.ny.nyc.columns
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.ny.nyc.dcp.rst b/docs/source/tasks.us.ny.nyc.dcp.rst
@@ -0,0 +1,7 @@
+tasks.us.ny.nyc.dcp module
+==========================
+
+.. automodule:: tasks.us.ny.nyc.dcp
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.ny.nyc.dob.rst b/docs/source/tasks.us.ny.nyc.dob.rst
@@ -0,0 +1,7 @@
+tasks.us.ny.nyc.dob module
+==========================
+
+.. automodule:: tasks.us.ny.nyc.dob
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.ny.nyc.package.rst b/docs/source/tasks.us.ny.nyc.package.rst
@@ -0,0 +1,7 @@
+tasks.us.ny.nyc.package module
+==============================
+
+.. automodule:: tasks.us.ny.nyc.package
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.ny.nyc.rst b/docs/source/tasks.us.ny.nyc.rst
@@ -0,0 +1,21 @@
+tasks.us.ny.nyc package
+=======================
+
+Submodules
+----------
+
+.. toctree::
+
+   tasks.us.ny.nyc.acris
+   tasks.us.ny.nyc.columns
+   tasks.us.ny.nyc.dcp
+   tasks.us.ny.nyc.dob
+   tasks.us.ny.nyc.package
+
+Module contents
+---------------
+
+.. automodule:: tasks.us.ny.nyc
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.ny.rst b/docs/source/tasks.us.ny.rst
@@ -0,0 +1,17 @@
+tasks.us.ny package
+===================
+
+Subpackages
+-----------
+
+.. toctree::
+
+    tasks.us.ny.nyc
+
+Module contents
+---------------
+
+.. automodule:: tasks.us.ny
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/tasks.us.rst b/docs/source/tasks.us.rst
@@ -8,6 +8,7 @@ Subpackages
 
     tasks.us.census
     tasks.us.epa
+    tasks.us.ny
 
 Submodules
 ----------

diff --git a/docs/source/testing.rst b/docs/source/testing.rst
@@ -11,28 +11,48 @@ ETL unit tests
 Unit tests ensure that there are no errors in the underlying utility classes
 that could cause errors in code you build on top of them.
 
-The tests also provide limited coverage for simple :ref:`tasks.util.ColumnTask`
-classes that don't need parameters set.
-
 Tests are run with:
 
+.. code:: shell
+
     make etl-unittest
 
+..
+Metadata integration tests
+--------------------------
+
+Integration tests make sure that the metadata being generated as part of your
+ETL will actually be queryable by the API.  For example, if you have an ETL
+that ingests data but does not 
+..
+
 API unit tests
 --------------
 
 API unit tests make sure the observatory-extension, which reads data and
 metadata from the ETL, are working right.
 
-``TODO``
+In order for this to function, you'll need to clone a copy of
+``observatory-extension`` into the root of the ``bigmetadata`` repo.
+
+.. code:: shell
+
+    git clone git@github.com:cartodb/observatory-extension
+    make extension-unittest
 
 Integration tests
 -----------------
 
 Integration tests ensure that the data from the ETL that is set for deployment
 is is able to return a measure for every piece of metadata.
 
-``TODO``
+As above, you'll need a copy of ``observatory-extension`` locally for this test
+to work.
+
+.. code:: shell
+
+    git clone git@github.com:cartodb/observatory-extension
+    make extension-autotest
 
 Diagnosing common issues in integration tests
 ---------------------------------------------