DEPRECATED. KA developers: this code has moved into the dataflow/log-export directory in the main webapp repository.
A Google Cloud Dataflow streaming job to export Khan Academy logs to BigQuery.
To get started, first follow the Cloud Dataflow instructions to set up your dependencies.
You will need a Java 8 JDK and Maven to build the project, and you need a properly-authenticated gcloud and bq on your PATH for deployment to work.
There are two make targets:
make deploy_testdeploys a dataflow job pointed at a temporary subscription and a test output table.make deploy_proddeploys a dataflow job pointed at the production subscription and output table.
See the documentation in the makefile for more details.