more documentation

tests/pdifftests/ENV.sh

@@ -1,3 +1,4 @@
+# See tests/pdifftests/README.md for details.
 export PORT=5001
 export DATABASE_URI='postgres://localhost/cycledash-dpxdt'
 export WEBHDFS_USER=testing

tests/pdifftests/README.md

@@ -0,0 +1,44 @@
+Notes on working with dpxdt tests
+---------------------------------
+
+There are some notes in [README.md][1] about how to run the dpxdt tests to
+check for changes. This document describes how to update and extend them.
+
+CycleDash requires a PostgreSQL database to run, and a rabbitmq workqueue to
+process new inputs. For the tests to be reproducible, it needs its own
+instances of each of these.
+
+Every time you run `dpxdt update`, it creates a fresh `cycledash-dpxdt` db
+using the schema from `schema.sql` and the data from
+`tests/pdifftests/data.sql`. This means that you must have postgres running to
+update the dpxdt tests, e.g. by running:
+
+    postgres -D /usr/local/var/postgres
+
+If a test is failing or you'd like to create a new test, you may find it
+helpful to run a CycleDash server using the dpxdt test DB. You can do this via:
+
+    gulp prod
+    ./tests/pdifftests/create-test-db.sh
+    ./tests/pdifftests/start-cycledash.sh
+
+And then visiting http://localhost:5001/. Note that the server is started
+without the Flask reloader, since this isn't necessary for automated testing.
+You'll have to modify `tests/pdifftests/ENV.sh` if you want that functionality.
+
+If you'd like to submit new runs, you'll need to start a separate workqueue and
+Celery workers. This can be done by running these commands:
+
+    brew install rabbitmq
+    /usr/local/opt/rabbitmq/sbin/rabbitmq-server
+    ./tests/pdifftests/worker.sh
+
+You can submit the new run using either the web UI or the the REST interface.
+
+To export the new run to the CSV file, you generate a database dump:
+
+    pg_dump --data-only cycledash-dpxdt > tests/pdifftests/data.sql
+
+For database migrations, it may be easiest to just resubmit all the runs.
+
+[1]: https://github.com/hammerlab/cycledash/blob/master/README.md

tests/pdifftests/create-test-db.sh

@@ -1,10 +1,7 @@
 #!/bin/bash
 # Creates a test database for use with dpxdt tests.
 #
-# If you'd like to update the CSV file which gets loaded into the DB, you can
-# run this script, use psql or cycledash to make your changes, and re-export
-# using:
-# pg_dump --data-only cycledash-dpxdt > tests/pdifftests/data.sql
+# See tests/pdifftests/README.md for details.
 
 set -o errexit
 

tests/pdifftests/start-cycledash.sh

@@ -1,6 +1,7 @@
 #!/bin/bash
 # Start a CycleDash server pointing at the dpxdt test DB.
 # Run this after running create-test-db.sh and "gulp prod"
+# See tests/pdifftests/README.md for details.
 set -o errexit
 
 source tests/pdifftests/ENV.sh

tests/pdifftests/worker.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# See tests/pdifftests/README.md for details.
 
 source tests/pdifftests/ENV.sh
 ./scripts/start-worker.sh