Continuous Integration

Our platform is contained in one monolithic repository that contains the code, configuration, and deployment setup for multiple services, including the main platform application, the PDF parsing service, serverless functions for event-oriented triggering, and the Dataflow pipelines for batch-oriented processing. With everything living in one repository, we can reduce the regression risk of any change to shared code by having a centralized location to run our continuous integration builds.

Travis CI

Travis CI runs our continuous integration build on the following events:

• The push of a new branch to the main remote or a fork of the repository
• The push of a new commit to an existing branch
• The issuance of a new pull request against the main remote
• The merging of a pull request into some branch (master or otherwise)
• The creation of a new tag/release (manual, through the Github UI, or otherwise)

Builds can also be triggered or replayed on demand.

The results of the builds are integrated into Github pull requests so that PRs should generally not be merged if a build is not passing. This can be overridden by a repository admin but should be avoided. The same core steps to the build can be run locally so developers should be in the habit of "running the build" prior to requesting further review on a PR, to save everyone time.

The README in the root of the repository contains badges for the Travis build and for our Coveralls code/test coverage. A break in the build that slips through to master is addressed as soon as it is identified.

Configuration

The Travis build is configured via the .travis.yml file in the root of the repository. This file includes:

1. Installing dependencies
2. Building the Docker image
3. Running "the build" (see below)
4. Caching the Docker image for later runs
5. Reporting coverage to Coveralls
6. Sending notifications to Slack when relevant

Additionally, the /travis folder in the root includes additional scripts for more complex commands, such as deploying to stage or exporting commits from our private repo to our public mirror (via Copybara).

The Build

"The build" is composed of the following steps, in order. The build short-circuits so a failure in an earlier step prevents processing of later steps. This is of course not necessarily true on local machines, so developers should be thorough locally even if there is a strange issue in a particular step.

1. Frozen set source validation
   1. This is a tool we created under recidiviz/tools/validate_source_modifications.py - certain sets of files should generally be changed together, such as the Python model for ingest_info.py and the Protobuf model for ingest_info.proto. If only a subset of files in a configured frozen set are changed, the build fails. This can be overridden by adding the text "[skip validation]" to the commit message.
2. mypy static type checking
3. pylint code linting (based on the Google Python Style Guide)
4. pytest all unit and integration tests
   1. The specific command includes --with-emulator which runs integration tests that rely on certain Google Cloud emulators, such as the Cloud Datastore emulator or the Pub/Sub emulator. These tests are not run when this flag is omitted.

Deploys to stage

When a new tag/release is created, this triggers a new Travis CI build that will run through all of the previously mentioned steps, and deploy the platform to our staging environment on success. The deploy script lives in /travis/deploy_staging.sh.

Coveralls

Travis and Coveralls are integrated via the build output: a code coverage report is output at the end of a successful run of tests, and the output is pushed up to Coveralls. From there, Coveralls results are processed and integrated into the pull request via comments like Travis. Coveralls' Github integration will mark code coverage as either pass or fail based on the delta between the previous percentage (e.g. on the branch this PR is being merged into) and the new state (i.e. the latest commit on this branch): if the coverage drops too far in a single commit, the build is marked as a failure to promote high test coverage.