[Security Solution][Rule Management] Getting rid off before hook if is not loading an archive #178891
Conversation
|
/ci |
MadameSheema
added
v8.14.0
Team:Detection Rule Management
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
There was a problem hiding this comment.
@MadameSheema thank you for addressing Cypress retry issue at Rules Management Area 🙏
The changes look good so impact it a bit unclear. Could you compare execution time of x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rules_table/rules_table_auto_refresh.cy.ts before and after the changes? It will give us a clear picture how much overhead we got recreating rules before each test.
Btw, createRule doesn't throw in case of failure since failOnStatusCode is set to false.
|
@maximpn please find below my responses
Yes, but if there is a problem with the rule creation the test itself is going to fail, in that case, Cypress retry is not going to execute the before hook, just code in beforeEach is reexecuted.
58s before my changes and 1:38s after my changes. Please let me know if that is acceptable or not. 
Even this might have impact in the execution time, it will give us more reliability in the future. |
banderror
added
test
test_ui_functional
Team:Detections and Resp
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
There was a problem hiding this comment.
@MadameSheema thank you for posting performance comparison 👍
I see the changes increase execution time twice (roughly speaking) but we still talk about max a few minutes (CI should be slower than local runs). It looks rather a trade off than an incremental improvement and since it's much more important for us to have Cypress retry functionality working reliably I approve your PR.
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
…s not loading an archive (elastic#178891) ## Summary Addresses: elastic#175022 Update: [Flaky test suite runner passed successfully](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5522) (cherry picked from commit 6f49ab8)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added
the
backport missing
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
1 similar comment
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
…ok if is not loading an archive (#178891) (#179333) # Backport This will backport the following commits from `main` to `8.13`: - [[Security Solution][Rule Management] Getting rid off before hook if is not loading an archive (#178891)](#178891) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Gloria Hornero","email":"gloria.hornero@elastic.co"},"sourceCommit":{"committedDate":"2024-03-25T11:04:13Z","message":"[Security Solution][Rule Management] Getting rid off before hook if is not loading an archive (#178891)\n\n## Summary\r\n\r\nAddresses: #175022 test suite runner passed\r\nsuccessfully](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5522)","sha":"6f49ab85e81fe123338a1a0a61266fb7c67606f9","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","test_ui_functional","release_note:skip","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","v8.14.0","v8.13.1"],"title":"[Security Solution][Rule Management] Getting rid off before hook if is not loading an archive","number":178891,"url":"#178891 Solution][Rule Management] Getting rid off before hook if is not loading an archive (#178891)\n\n## Summary\r\n\r\nAddresses: #175022 test suite runner passed\r\nsuccessfully](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5522)","sha":"6f49ab85e81fe123338a1a0a61266fb7c67606f9"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/178891","number":178891,"mergeCommit":{"message":"[Security Solution][Rule Management] Getting rid off before hook if is not loading an archive (#178891)\n\n## Summary\r\n\r\nAddresses: #175022 test suite runner passed\r\nsuccessfully](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5522)","sha":"6f49ab85e81fe123338a1a0a61266fb7c67606f9"}},{"branch":"8.13","label":"v8.13.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Gloria Hornero <gloria.hornero@elastic.co> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
kibanamachine
removed
the
backport missing
Summary
Resolves: #175022
Update:
Flaky test suite runner passed successfully