You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a follow up epic to #7316 to increase the dimensions we test on - i.e. the environments we test in, regarding which dependencies are used, different configurations etc. The documentation for the current state of automated test can be found here: internal.
Context
After having the basic framework for automated testing of self-hosted gitpod within different environments in place, we want to increase our test coverage regarding the environments we test on. This should allow us to further booster confidence for a larger set of customers. and reduce the time it takes for us to test manually.
Value
Increased Confidence due to higher test coverage: Due to the bring-your-own "hardware" and dependencies nature of Self-Hosted, it will be running in many different environments and configurations. As such, we need to ensure that any changes we make to Gitpod run smoothly in a large part of these environments - at the very least on our reference architecture. Increasing the amount of configurations we test on will help with this
The more we can test in an automated manner, the less we have to test manually. This means more time to focus on feature work.
Acceptance Criteria
Tests run on all of our reference architectures, and all variations of them
We test for at least 70% of the possible testing dimensions laid out in this (internal) document.
Time to release a self-hosted release is reduced to half a day
Measurement
We can map our test coverage to our internal list of possible testing dimensions in order to roughly see how large our coverage (regarding what we test on, not what we test) is.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Summary
This is a follow up epic to #7316 to increase the dimensions we test on - i.e. the environments we test in, regarding which dependencies are used, different configurations etc. The documentation for the current state of automated test can be found here: internal.
Context
After having the basic framework for automated testing of self-hosted gitpod within different environments in place, we want to increase our test coverage regarding the environments we test on. This should allow us to further booster confidence for a larger set of customers. and reduce the time it takes for us to test manually.
Value
Acceptance Criteria
Measurement
Issues
Increase test coverage and ensure tests pass:
Terraform Tooling:
terraform
module with reference architecture #11533terraform
module with reference architecture #11534terraform
module with reference architecture #11535Increase infra coverage:
Other issues:
The text was updated successfully, but these errors were encountered: