-
Notifications
You must be signed in to change notification settings - Fork 8.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Obs AI Assistant] Fix flaky e2e test (contextual insights for APM errors) #184642
Conversation
Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services) |
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6187[❌] x-pack/test/observability_ai_assistant_functional/enterprise/config.ts: 24/100 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6188[❌] x-pack/test/observability_ai_assistant_functional/enterprise/config.ts: 0/25 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6189[❌] x-pack/test/observability_ai_assistant_functional/enterprise/config.ts: 0/25 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6191[❌] x-pack/test/observability_ai_assistant_functional/enterprise/config.ts: 0/25 tests passed. |
buildkite test this |
1 similar comment
buildkite test this |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6202[❌] x-pack/test/observability_ai_assistant_functional/enterprise/config.ts: 0/25 tests passed. |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#6204[✅] x-pack/test/observability_ai_assistant_functional/enterprise/config.ts: 25/25 tests passed. |
async function openContextualInsights() { | ||
await retry.try(async () => { | ||
await testSubjects.click(ui.pages.contextualInsights.button); | ||
const isOpen = | ||
(await ( | ||
await find.byCssSelector(`[aria-controls="${ui.pages.contextualInsights.container}"]`) | ||
).getAttribute('aria-expanded')) === 'true'; | ||
expect(isOpen).to.be(true); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was the culprit of the failure. The accordion was clicked before it was ready/interactive. This is somewhat ugly but feels robust.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: i always found retry.try
to be slow as it takes the default config timeout which is of 30 sec making the test fails very late.
There is an alternate called
await retry.tryForTime(TIME_IN_MILLISECONDS, async () => {});
With this, i limit my test to 5*1000ms which makes failures faster. Its not a hard rule, sometimes tests would need more time, but the default time of 30 seconds definitely makes me feel that something is wrong with the test.
buildkite test this |
…ional-test # Conflicts: # packages/kbn-apm-synthtrace-client/src/lib/apm/apm_error.ts
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#6389[✅] x-pack/test/observability_ai_assistant_functional/enterprise/config.ts: 25/25 tests passed. |
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
Async chunks
History
To update your PR or re-run it, just comment with: |
fixes [elastic#176948](elastic#176948) fixes [elastic#177061](elastic#177061) fixes [elastic#177104](elastic#177104) ## Summary I managed to reproduce the problem when `error_grouping_key` was null. It was probably fixed here elastic#184642 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6512 Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
fixes [elastic#176948](elastic#176948) fixes [elastic#177061](elastic#177061) fixes [elastic#177104](elastic#177104) ## Summary I managed to reproduce the problem when `error_grouping_key` was null. It was probably fixed here elastic#184642 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6512 Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit 5df7bb8)
# Backport This will backport the following commits from `main` to `8.15`: - [[APM] Unskip APM alerting tests (#188006)](#188006) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Carlos Crespo","email":"crespocarlos@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-07-11T11:30:35Z","message":"[APM] Unskip APM alerting tests (#188006)\n\nfixes [#176948](#176948 [#177061](#177061 [#177104](#177104 Summary\r\n\r\nI managed to reproduce the problem when `error_grouping_key` was null.\r\nIt was probably fixed here #184642: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"5df7bb87c0a9475d7f860fb45437c4624a2bd0e5","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services","v8.16.0"],"title":"[APM] Unskip APM alerting tests","number":188006,"url":"#188006 Unskip APM alerting tests (#188006)\n\nfixes [#176948](#176948 [#177061](#177061 [#177104](#177104 Summary\r\n\r\nI managed to reproduce the problem when `error_grouping_key` was null.\r\nIt was probably fixed here #184642: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"5df7bb87c0a9475d7f860fb45437c4624a2bd0e5"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"#188006 Unskip APM alerting tests (#188006)\n\nfixes [#176948](#176948 [#177061](#177061 [#177104](#177104 Summary\r\n\r\nI managed to reproduce the problem when `error_grouping_key` was null.\r\nIt was probably fixed here #184642: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"5df7bb87c0a9475d7f860fb45437c4624a2bd0e5"}}]}] BACKPORT--> Co-authored-by: Carlos Crespo <crespocarlos@users.noreply.github.com>
Closes #184071 #184029
Fixes flaky e2e tests
error.grouping_key
. This required a small change in synthtrace.