[10k]: Back-end CI/CD

[widal001]

Description

Configure a CI/CD pipeline to automate code quality checks and deploys for the backend codebase.

Business Value & Goals

The purpose of this body of work is to:

• Ensure that we are developing the proper infrastructure early on so that we can increase our delivery speed in the long run
• Enforce key code quality and security standards with every commit that is pushed to our database
• Reduce the likelihood and the impact of manual errors being introduced by automating deploys and enabling rollbacks of failed builds

Ideally, we would like to ensure a certain percentage of our ongoing sprint velocity (at least 20-30% at the beginning?) is devoted to improving our developer tools (i.e., auto-formatters, build tools, etc) and communication tools (i.e., our wiki, milestones docs, etc). By investing in speeding up team productivity, we will drastically increase the speed of our delivery.

We should track the percentage of story points devoted to improving team efficiency and joy and ensure that they're hitting at least our target percentage each sprint.

User Stories

As a project maintainer, I want:

• most of the critical development infrastructure to be in place when we officially launch the API, so that we can deploy bug fixes or new features quickly once the API is live
• automated checks to run against PRs submitted by open source contributors to ensure that it meets our code quality and security standards
• automated tests to run in each development environment that we deploy to, so that we can catch potential build failures before or shortly after we deploy
• deploys that result in failed builds or that violate key checks to support easy rollbacks to the previous stable release, so that we don't need to spend time manually recovering from a deployment failure

Open Questions

For discussion: is a STAGING environment and staging branch necessary at this stage, or should we continuously deploy to PROD as feature branches are merged into main?

Dependencies

• [10k]: API Planning #42
• [10k]: DB planning #48
• [10k]: Developer tools planning #50
• [10k]: Security approval for static site launch #53

Sub-Tasks

TO DO

Definition of Done

• When commits are made to any branch, these should automatically:
  • Run lint suite
  • Run test suite
  • Deploy branch to backend and/or frontend test environments when merging to main
• When commits are made to staging (or dev) branch, they should automatically:
  • Deploy branch to STAGING (or DEV) environments
  • Run smoke tests and integration tests in STAGING (or DEV) - will be covered in [10k]: Integration testing #763
• When merges are made to main (or a dedicated prod branch), these should automatically:
  • Deploy branch to PROD environments through tags
  • Run smoke tests and integration tests in PROD - will be covered in [10k]: Integration testing #763
• When deploys or critical checks fail:
  • The branch should roll back to the previous stable release with minimal intervention. See comment below
  • Merge would not happen if deploy fails so that engineer can fix the code
  • This stable release should automatically deploy to the corresponding environment
• For each of our environments we are capturing data on the following metrics in Github Actions:
  • Frequency of deploys
  • Duration of time it takes per deploy
  • Duration of time it takes per rollback

[sumiat]

• Team recommends pulling out integration into its own integration testing 10K. Created here: [10k]: Integration testing #763
• For the DoD of the branch rolling back to the previous stable release with minimal intervention - the team cautions against this approach. Currently, it will not deploy and fixes would need to be made before deployment.