-
Notifications
You must be signed in to change notification settings - Fork 847
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[META] CI Improvement Plan #4019
Labels
Comments
8 tasks
Device matrix testing #4803 |
8 tasks
kavilla
added a commit
to kavilla/OpenSearch-Dashboards-1
that referenced
this issue
Sep 27, 2023
Utilizing the GitHub `workflow_dispatch`, we can trigger a job with a specific inputs. By default, we run from this repo: https://github.com/opensearch-project/opensearch-dashboards-functional-test This workflow uses the target branch of the PR to pull down from the FTRepo and run the tests from that specific branch. For example, PRs against `main` will pull down `main` from the FTRepo. The problem occurs when new functionality is opened or a bug is fixed and, per industry standard, we want to see tests added to ensure functionality, stability, and raise the bar. However, the cypress tests PR into the FTRepo depends on the new code to be merged for the CI within the FTRepo to work. So we have a stalemate that usually slows down PR review time. Here, a manual run can be triggered by a maintainer. It will utilize the source branch from the PR if provided. Related to: opensearch-project#4019 Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
kavilla
added a commit
that referenced
this issue
Sep 27, 2023
Utilizing the GitHub `workflow_dispatch`, we can trigger a job with a specific inputs. By default, we run from this repo: https://github.com/opensearch-project/opensearch-dashboards-functional-test This workflow uses the target branch of the PR to pull down from the FTRepo and run the tests from that specific branch. For example, PRs against `main` will pull down `main` from the FTRepo. The problem occurs when new functionality is opened or a bug is fixed and, per industry standard, we want to see tests added to ensure functionality, stability, and raise the bar. However, the cypress tests PR into the FTRepo depends on the new code to be merged for the CI within the FTRepo to work. So we have a stalemate that usually slows down PR review time. Here, a manual run can be triggered by a maintainer. It will utilize the source branch from the PR if provided. Related to: #4019 Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
opensearch-trigger-bot bot
pushed a commit
that referenced
this issue
Sep 27, 2023
Utilizing the GitHub `workflow_dispatch`, we can trigger a job with a specific inputs. By default, we run from this repo: https://github.com/opensearch-project/opensearch-dashboards-functional-test This workflow uses the target branch of the PR to pull down from the FTRepo and run the tests from that specific branch. For example, PRs against `main` will pull down `main` from the FTRepo. The problem occurs when new functionality is opened or a bug is fixed and, per industry standard, we want to see tests added to ensure functionality, stability, and raise the bar. However, the cypress tests PR into the FTRepo depends on the new code to be merged for the CI within the FTRepo to work. So we have a stalemate that usually slows down PR review time. Here, a manual run can be triggered by a maintainer. It will utilize the source branch from the PR if provided. Related to: #4019 Signed-off-by: Kawika Avilla <kavilla414@gmail.com> (cherry picked from commit 34a8594) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> # Conflicts: # CHANGELOG.md
kavilla
added a commit
to kavilla/OpenSearch-Dashboards-1
that referenced
this issue
Sep 28, 2023
Originally: opensearch-project#5134 I believed it to be a good idea to ensure that triggering the CI should always run a batch of tests. But I realized that some tests can be added and just be bad tests until updated. Or if we want to verify a flaky test in the CI. So instead of only making it additional tests, the spec input will complete replace the default tests (unless you append to the default). Empty will run the default spec still. This will append on to the PR comment with the spec that was run. So at least maintainers can see it passed for a set of tests but it did not run the ones we usually run. Also adding some formatting for empty values for the PR comment. Issue related: opensearch-project#4019 Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
kavilla
added a commit
that referenced
this issue
Sep 29, 2023
Originally: #5134 I believed it to be a good idea to ensure that triggering the CI should always run a batch of tests. But I realized that some tests can be added and just be bad tests until updated. Or if we want to verify a flaky test in the CI. So instead of only making it additional tests, the spec input will complete replace the default tests (unless you append to the default). Empty will run the default spec still. This will append on to the PR comment with the spec that was run. So at least maintainers can see it passed for a set of tests but it did not run the ones we usually run. Also adding some formatting for empty values for the PR comment. Issue related: #4019 Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
8 tasks
opensearch-trigger-bot bot
pushed a commit
that referenced
this issue
Oct 2, 2023
Originally: #5134 I believed it to be a good idea to ensure that triggering the CI should always run a batch of tests. But I realized that some tests can be added and just be bad tests until updated. Or if we want to verify a flaky test in the CI. So instead of only making it additional tests, the spec input will complete replace the default tests (unless you append to the default). Empty will run the default spec still. This will append on to the PR comment with the spec that was run. So at least maintainers can see it passed for a set of tests but it did not run the ones we usually run. Also adding some formatting for empty values for the PR comment. Issue related: #4019 Signed-off-by: Kawika Avilla <kavilla414@gmail.com> (cherry picked from commit 6154e2d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kavilla
pushed a commit
that referenced
this issue
Oct 2, 2023
Originally: #5134 I believed it to be a good idea to ensure that triggering the CI should always run a batch of tests. But I realized that some tests can be added and just be bad tests until updated. Or if we want to verify a flaky test in the CI. So instead of only making it additional tests, the spec input will complete replace the default tests (unless you append to the default). Empty will run the default spec still. This will append on to the PR comment with the spec that was run. So at least maintainers can see it passed for a set of tests but it did not run the ones we usually run. Also adding some formatting for empty values for the PR comment. Issue related: #4019 (cherry picked from commit 6154e2d) Signed-off-by: Kawika Avilla <kavilla414@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We'd like to improve our CI for OpenSearch Dashboards. Creating this issue to start forming the plan.
Initial AI: baseline metrics to measure progress against and establish goal.
CI Labelled Issues
The text was updated successfully, but these errors were encountered: