Goodbye Travis (#7100)

broadinstitute · Mar 24, 2023 · 209f67d · 209f67d
1 parent 8b7f702
commit 209f67d
Show file tree

Hide file tree

Showing 10 changed files with 78 additions and 142 deletions.
diff --git a/README.md b/README.md
@@ -1,4 +1,3 @@
-[![Build Status](https://travis-ci.com/broadinstitute/cromwell.svg?branch=develop)](https://travis-ci.com/broadinstitute/cromwell?branch=develop)
 [![codecov](https://codecov.io/gh/broadinstitute/cromwell/branch/develop/graph/badge.svg)](https://codecov.io/gh/broadinstitute/cromwell)
 
 ## Welcome to Cromwell

diff --git a/docs/developers/Centaur.md b/docs/developers/Centaur.md
@@ -14,9 +14,11 @@ You can now run the tests from another terminal.
 
 There are two ways to invoke the integration tests:
 
-* `sbt "centaur / IntegrationTest / test"` - compiles and run via sbt directly, simple but also has the problem of running 2x cores tests in parallel which can overwhelm your Cromwell server if running in a development environment
+* `sbt "centaur / IntegrationTest / test"` - compiles Centaur and runs all tests via sbt directly. Tests are expected to be in the `centaur/src/main/standardTestCases` directory. This can be changed by modifying `reference.conf`.
 
-* `src/ci/bin/testCentaurLocal.sh` - runs the same tests using the continuous integration pipeline configuration
+* `src/ci/bin/testCentaurLocal.sh` - runs the same tests using the continuous integration pipeline configuration.
+
+* Tests that require different Cromwell and Centaur configurations can be invoked by calling the various scripts in `src/ci/bin`.
 
 ### Tags
 
@@ -104,3 +106,71 @@ In case the absolute path the cromwell root is used (for example: `/home/my_user
 
 In case testing of the caching is required `<<CACHE_HIT_UUID>>` can be used. 
 The testFormat should be `runtwiceexpectingcallcaching`.
+
+
+## Centaur Test Types
+Both Cromwell and Centaur require configuration files in order to correctly build and test various parts of the system. Because of this, we divide
+our tests into groups depending on which configuration files they require. Below is the current matrix of configuration files and test source directories.
+
+## Upgrade / Horicromtal / etc.
+
+| CI Test Type                  | Cromwell Config                                                  | Centaur Config                                         |
+|-------------------------------|------------------------------------------------------------------|--------------------------------------------------------|
+| Engine Upgrade                | `(backend)_application.conf`                                     | `centaur_application.conf`*                            |
+| Horicromtal                   | `papi_[v2beta or v2alpha1]_horicromtal_application.conf`**       | `centaur_application_`<br>`horicromtal.conf`           |
+| Horicromtal<br>Engine Upgrade | `papi_v2beta_application.conf`**                                 | `centaur_application_`<br>`horicromtal_no_assert.conf` |
+| PAPI Upgrade                  | `papi_v1_v2alpha1_upgrade_application.conf`**                    | `centaur_application.conf`*                            |
+| Papi Upgrade<br>New Workflows | `(backend)_application.conf`                                     | `centaur_application.conf`*                            |
+| Azure Blob                    | `centaur_blob_test.conf`**                                       | `centaur_application.conf`*                            |
+| WDL Upgrade                   | `(backend)_application.conf`                                     | `centaur_application.conf`*                            |
+| (other)                       | `(backend)_application.conf`                                     | `centaur_application.conf`*                            |
+
+| CI Test Type                  | ScalaTest Spec              | Test Directory                      |
+|-------------------------------|-----------------------------|-------------------------------------|
+| Engine Upgrade                | `EngineUpgradeTestCaseSpec` | `engineUpgradeTestCases`            |
+| Horicromtal                   | `CentaurTestSuite`          | `standardTestCases`***              |
+| Horicromtal<br>Engine Upgrade | `EngineUpgradeTestCaseSpec` | `engineUpgradeTestCases`***         |
+| PAPI Upgrade                  | `PapiUpgradeTestCaseSpec`   | `papiUpgradeTestCases`              |
+| PAPI Upgrade<br>New Workflows | `CentaurTestSuite`          | `papiUpgradeNewWorkflowsTestCases`  |
+| Azure Blob                    | `CentaurTestSuite      `    | `azureBlobTestCases`                |
+| WDL Upgrade                   | `WdlUpgradeTestCaseSpec`    | `standardTestCases`****             |
+| (other)                       | `CentaurTestSuite`          | `standardTestCases`                 |
+
+<small>
+\* Centaur Config always uses `centaur_application.conf` except when overridden with `papi_v2alpha1_centaur_application.conf`
+or `papi_v2beta_centaur_application.conf`
+  ([48 preview link](https://github.com/broadinstitute/cromwell/blob/a7d0601/src/ci/bin/test.inc.sh#L455-L457))  
+\*\* Cromwell Config overrides
+  ([47 link](https://github.com/broadinstitute/cromwell/blob/47/src/ci/bin/test.inc.sh#L213-L221))  
+\*\*\* Test Directory overrides
+  ([47 link](https://github.com/broadinstitute/cromwell/blob/47/src/ci/bin/test.inc.sh#L440-L449))  
+\*\*\*\* Test Directory only tests tagged with `wdl_upgrade`
+  ([47 link](https://github.com/broadinstitute/cromwell/blob/47/centaur/src/main/resources/standardTestCases/write_lines.test#L3))  
+</small>
+
+- Engine Upgrade: Retrieves the [Cromwell Version](https://github.com/broadinstitute/cromwell/blob/47/project/Version.scala#L8) then retrieves the previous jar/docker-image from DockerHub. Centaur starts with the prior version, then restarts with the compiled source code.
+- Horicromtal: Runs a [docker-compose](https://github.com/broadinstitute/cromwell/blob/47/src/ci/docker-compose/docker-compose-horicromtal.yml) with:
+    1. db-mstr: started first
+    2. sum-back: runs summarizer
+    3. front-back: exposes HTTP
+- Horicromtal Engine Upgrade: Combination of Horicromtal and Engine Upgrade
+- PAPI Upgrade: Tests run with an older version of Papi and upon restart use a newer version of Papi
+- PAPI Upgrade New Workflows: Test definition [does not run any tests](https://travis-ci.org/broadinstitute/cromwell/jobs/475378412)
+- WDL Upgrade: Upgrades WDL from draft-2 to 1.0 before testing
+- (other): Runs `*.test` files listing the configured backend names
+
+## RDBMS
+
+| Backend | MySQL  | PostgreSQL  | MariaDB  |
+|---------|:------:|:-----------:|:--------:|
+| AWS     |   ✅   |             |          |
+| Local   |   ✅   |      ✅     |          |
+| PAPI V2 |   ✅   |             |    ⭕    |
+| SLURM   |   ✅   |             |          |
+| TES     |   ✅   |             |          |
+
+<small>
+⭕ Tests Horicromtal Engine Upgrade versus standard Centaur suite
+</small>
+
+All backends run against MySQL. The Local backend also test PostgreSQL, allowing contributors ensure WDLs work with PostgreSQL. MariaDB is tested on a specialized upgrade, where the MySQL connector client is used first, and the MariaDB client is used after restart.
diff --git a/docs/developers/Contribute.md b/docs/developers/Contribute.md
@@ -8,13 +8,6 @@ First off, here are useful links:
 * Source on Github: [github.com/broadinstitute/cromwell](https://github.com/broadinstitute/cromwell/tree/develop/docs)
 * Builds on ReadTheDocs: [readthedocs.org/projects/cromwell](https://readthedocs.org/projects/cromwell/builds/)
 * How to build and view the documentation locally: [mkdocs.readthedocs.io](https://mkdocs.readthedocs.io/en/stable/#installation)
-* Public Google Docs - Table of Contents:  [drive.google.com/open?id=1myTVzWx5HG720nPBUAF9vE8XTacrzCW70RpgsjMnchM](https://drive.google.com/open?id=1myTVzWx5HG720nPBUAF9vE8XTacrzCW70RpgsjMnchM)
-
-### "needs docs"
-
-There are plenty of areas of the Cromwell documentation that need to be updated, improved, or added. Within the Cromwell repo on Github there are many issues labeled as ["needs docs"](https://github.com/broadinstitute/cromwell/issues?q=is%3Aopen+is%3Aissue+label%3A%22needs+docs%22), so feel free to start there. 
-
-If you would like to request additional documentation, you can create a [Github issue in the Cromwell repo](https://github.com/broadinstitute/cromwell/issues/new).
 
 ### Writing Tips
 

diff --git a/docs/developers/bitesize/ci/CICD.md b/docs/developers/bitesize/ci/CICD.md
diff --git a/docs/developers/bitesize/ci/CaaS_DEV_CD.svg b/docs/developers/bitesize/ci/CaaS_DEV_CD.svg
diff --git a/docs/developers/bitesize/ci/Cromwell_Deployment_Strategies.svg b/docs/developers/bitesize/ci/Cromwell_Deployment_Strategies.svg
diff --git a/docs/developers/bitesize/ci/travis_centaur.md b/docs/developers/bitesize/ci/travis_centaur.md
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -102,8 +102,6 @@ nav:
         - Horizontal Cromwell ("Horicromtal"): developers/bitesize/horicromtal.md
         - General Topics:
             - Simpletons: developers/bitesize/general/simpletons.md
-        - Continuous Integration (CI) / Continuous Delivery (CD): developers/bitesize/ci/CICD.md
-        - Travis Centaur: developers/bitesize/ci/travis_centaur.md
 
 theme: readthedocs
 
@@ -112,5 +110,5 @@ extra_css:
 extra_javascript:
     - js/extra.js
 
-copyright: Copyright &copy; 2019 <a href="https://broadinstitute.org">Broad Institute</a>
+copyright: Copyright &copy; 2023 <a href="https://broadinstitute.org">Broad Institute</a>
 # google_analytics: ['identifierID', 'URL.org']
diff --git a/processes/external-contributions/README.MD b/processes/external-contributions/README.MD
@@ -38,28 +38,14 @@ Ask the questions:
 
 ### Run CI against the PR
 
-Note: inspired by the community answer [here](https://github.community/t5/How-to-use-Git-and-GitHub/Checkout-a-branch-from-a-fork/td-p/77).
-
-- Problem: our CI will only run against branches of the `broadinstitute/cromwell` repo submitted by team members.
-- To turn a community contribution into a PR that travis will run against:
-  - Run the `scripts/make_pr_clone_branch.sh` script, providing the github PR number of the external PR as the command line argument
-    - Eg: `scripts/make_pr_clone_branch.sh 5333` to make a PR clone branch for PR #5333.  
-  - Push the branch to github
-    - Note: The final `git push` command may not work as-is, depending on your ~/.gitconfig value of `push.default`. If it doesn't work then one of the following solutions may work:
-      - Setting `git`'s `push.default` config value to be '`current`'.
-      - Using `git push --set-upstream origin 5070_pr_clone` instead
-      - Using `git push origin HEAD` instead
-  - Create a new PR for the clone branch. Indicate that it only exists to test a community contribution. 
-    - Example title: `[PR 5333 Clone] PR for CI only`
-- If you need to re-sync the clone PR against changes on the remote branch, the same script will *update* your local reference allowing you to force-push changes to the same github branch!  
+- Opening a PR triggers CI to run automatically. However, if the PR was not opened by a trusted contributor, it will wait in a Github Actions queue until a trusted contributor explicitly allows CI tests run.
 
 ### Cycle through Review and CI
 
 - If the community contributor makes changes following your reviews or the CI results:
   - Glance at the changes to make sure they still seem reasonable.
   - Make any additional comments
-  - Re-fetch the remote reference: `git fetch -f origin refs/pull/860/head:938_pr_clone`
-  - Push the changes back up to github to re-trigger the CI on your clone PR.
+  - Rerun CI.
 
 ### Merge the PR