-
Notifications
You must be signed in to change notification settings - Fork 22
Testing and QA workflow
After a PR has been approved, the testing stage begins. The process for testing (after migrating to continuous delivery) is as follows.
For PRs that will be merged to master (either directly or through a temporary branch), the default procedure is to add the "testing" label to the PR. This will execute the Cypress tests against applicable backend versions and deploy a build to Netlify. Applicable backend versions in this context is all backend versions the app is compatible with, except the latest dev version (the tests are already running against dev when pushing new commits).
The app will be deployed to Netlify no matter if the tests are successful or not. It will be up to the dev to decide whether to wait for successful Cypress tests before moving on to step 2 (in theory it makes sense to only deploy up on successful Cypress testing, but we opted not to add another wait period for the dev here. The failed tests will be caught eventually).
For backport PRs, you should add the "preview" label to deploy to Netlify (there is no additional backends to run Cypress tests against).
After all PRs have been dealt with in the previous step (including backports), move the JIRA-ticket to testing and assign it to QA (Geetha).
QA will assign the ticket back to the dev and add a comment (about the successful testing). Dev will merge the PRs (including backports) and finally set the JIRA-ticket to “Done”.
If QA finds that an update is needed, they will set the JIRA-ticket to “Needs update” and assign the ticket to the dev. Dev will make the required fixes and restart from step 1 (with the difference being that the labels will have to be removed and re-added to trigger a workflow run and the JIRA-ticket should be moved to "retesting" instead of "testing")
Dependabot PRs: Normally we don't move these through QA, but we should run Cypress tests on them. Transfix PRs: Automatically merged without additional Cypress- or QA-testing.
In addition, some PRs that don't touch production code can be exempted from testing (see skip-e2e-tests below). We will use the devs judgement here.
There are some optional labels that can be used.
e2e-tests: Will trigger Cypress tests only.
skip-e2e-tests: You can choose to opt-out from Cypress testing. You will NOT normally do this, but there are some valid use-cases. For example, if up on running successful tests you push a new commit, the successful test results will be invalidated. If you only made a minor change to non-production code, it might not be useful to rerun all the tests.
preview: The default label used for backports, but also available when merging to master. Will deploy to Netlify.