Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/group1/tests/alerting/backfill/find·ts - alerting api integration security and spaces enabled Alerts - Group 1 alerts backfill rule runs find backfill space_1_all_alerts_none_actions at space1 should handle finding backfill requests with query string appropriately #181907

Closed
kibanamachine opened this issue Apr 26, 2024 · 1 comment · Fixed by #181868
Closed

Failing test: X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/group1/tests/alerting/backfill/find·ts - alerting api integration security and spaces enabled Alerts - Group 1 alerts backfill rule runs find backfill space_1_all_alerts_none_actions at space1 should handle finding backfill requests with query string appropriately #181907

kibanamachine opened this issue Apr 26, 2024 · 1 comment · Fixed by #181868
Assignees
@ymao1
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@kibanamachine
Copy link
Contributor

A test failed on a tracked branch

Error: expected 0 to sort of equal 1
    at Assertion.assert (expect.js:100:11)
    at Assertion.eql (expect.js:244:8)
    at Context.<anonymous> (find.ts:466:67)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:73:16) {
  actual: '0',
  expected: '1',
  showDiff: true
}

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Apr 26, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Apr 26, 2024
@kibanamachine kibanamachine added the Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) label Apr 26, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Apr 26, 2024
@ymao1 ymao1 self-assigned this Apr 30, 2024
@ymao1 ymao1 mentioned this issue Apr 30, 2024
Merged
@ymao1 ymao1 closed this as completed in 717f5d6 Apr 30, 2024
yuliacech pushed a commit to yuliacech/kibana that referenced this issue May 3, 2024
@ymao1 @kibanamachine @yuliacech
Fixes flaky backfill tests (elastic#181868)
7ecf54b 
Resolves elastic#181778
Resolves elastic#181861
Resolves elastic#181862
Resolves elastic#181875
Resolves elastic#181840
Resolves elastic#181893
Resolves elastic#182029
Resolves elastic#181907



## Summary

The schedule request tries to schedule backfills for multiple rule
types. The test was testing for a 403 Forbidden message containing a
specific rule type ID (the first one in the schedule request) but in
practice, the rule type that the authorization check runs on is
non-deterministic so could be any of the rule type IDs in the schedule
request. Changed the test to match the error pattern instead.

For the `GET` and `FIND` tests, we're getting 404s when we're expecting
200s so it seems like the backfill runs are completing before the get &
find request are issued. I increased the schedule for the backfill runs
so the jobs don't finish and get deleted before the tests run. I also
changed the pattern for matching `status` to allow for something other
than `pending` status (for backfill schedules that complete before the
GET request is made)

Flaky test runner runs: 
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5804
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5806

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ymao1 ymao1

Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
No open projects
AppEx: ResponseOps - Execution & Connectors
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixes flaky backfill tests ymao1/kibana
3 participants
@ymao1 @elasticmachine @kibanamachine