Failing test: X-Pack Search Sessions Integration.x-pack/test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time·ts - Dashboard save a search sessions with relative time Saves and restores a session with relative time ranges #97701

kibanamachine · 2021-04-20T18:24:11Z

A test failed on a tracked branch

Error: retry.try timeout: Error: expected 9 to equal 11
    at Assertion.assert (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/expect/expect.js:227:8)
    at Assertion.be (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/expect/expect.js:69:22)
    at /dev/shm/workspace/parallel/4/kibana/test/functional/services/dashboard/expectations.ts:210:43
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at runAttempt (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:27:15)
    at retryForSuccess (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:66:21)
    at Retry.try (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry.ts:32:14)
    at DashboardExpect.savedSearchRowCount (/dev/shm/workspace/parallel/4/kibana/test/functional/services/dashboard/expectations.ts:205:7)
    at checkSampleDashboardLoaded (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:96:5)
    at Context.<anonymous> (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:68:7)
    at Object.apply (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)
    at onFailure (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:57:13)
    at Retry.try (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry.ts:32:14)
    at DashboardExpect.savedSearchRowCount (/dev/shm/workspace/parallel/4/kibana/test/functional/services/dashboard/expectations.ts:205:7)
    at checkSampleDashboardLoaded (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:96:5)
    at Context.<anonymous> (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:68:7)
    at Object.apply (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)

First failure: Jenkins Build

The text was updated successfully, but these errors were encountered:

elasticmachine · 2021-04-20T18:28:58Z

Pinging @elastic/kibana-presentation (Team:Presentation)

tylersmalley · 2021-04-20T18:30:27Z

Currently only three failures in the past week:

kibanamachine · 2021-07-18T01:07:45Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T01:07:46Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T01:27:18Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T01:27:19Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T01:52:14Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T02:09:12Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T02:53:23Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T03:10:01Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T03:51:54Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T04:13:47Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T04:54:49Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T05:10:58Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T05:52:59Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T06:11:19Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T06:58:23Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T07:09:41Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T07:52:41Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T08:10:16Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T08:51:30Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T09:09:03Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T09:52:46Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T10:09:46Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T10:53:42Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T11:10:00Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T16:11:42Z

New failure: Jenkins Build

kibanamachine · 2021-07-18T16:56:48Z

New failure: Jenkins Build

elasticmachine · 2021-07-19T08:53:21Z

Pinging @elastic/kibana-app-services (Team:AppServices)

Dosant · 2021-07-19T12:00:45Z

A recent change that might have indirectly caused these failures: #104780

Looks like test failures aren't related to the search sessions, but all the tests just fail to load a flights sample dashboard. They report: "15 of 17 panels loaded".

These two panels in "no results state" are causing the problem, as they don't set a proper of data- attributes to let test suite know that they finished loading.

I filed #106085 and as soon as this is fixed by @elastic/kibana-app , we can re-enable the test.

@wylieconlon, I am also not sure if this is a valid behaviour that those visualizations could produce a "No results" state with sample data. I assume you didn't hit this in the pr #104780 ? I am not sure if this is some kind of a race condition in lens code or if this is a valid possible behaviour with how sample data works. Do you maybe know? Otherwise, I think it worth creating an issue to investigate and make it clear?

Also a note: today the skipped test is passing, both locally and on ci: #106071 (comment)

wylieconlon · 2021-07-19T20:26:37Z

@Dosant Yes I think you're right that #106085 is the cause, but there's something else that's special about those two panels: they are both using the time shift feature. Could that be related, since the data for all the other panels is loaded?

Dosant · 2021-07-20T08:56:11Z

Could that be related, since the data for all the other panels are loaded?

I am not sure, but an important remark is that the empty state happens not when the test tries to restore a search session, but it happens and fails on the initial dashboard load.
So this could likely happen in any other test that would use sample flights data and this empty state is not related to search sessions functionality

kibanamachine · 2023-02-14T15:00:11Z

New failure: CI Build - main

kibanamachine · 2023-03-20T15:53:00Z

New failure: CI Build - 8.7

ThomThomson · 2023-03-21T15:26:57Z

It looks like this is a completely different failure this time:

delayed vs 1 week earlier and canceled vs 1 week earlier both loaded successfully:
The error this time is: The element [data-test-subj="embeddableError"] was still present when it should have disappeared.

My guess is that one of the charts on this dashboard errored out, but it's difficult to tell which one at this point.

I think the next step is to replace instances of await testSubjects.missingOrFail('embeddableError'); with a method on the dashboard service that can actually report which panels have failed, then run a flaky test runner against this.

Closes #97701 ## Summary This PR replaces all instances of `await testSubjects.missingOrFail('embeddableError');` with a call to the new `noErrorEmbeddablesPresent()` method from the `dashboardExpect` service. This `noErrorEmbeddablesPresent()` method adds some extra debugging logic to the error embeddable check where, if at least one error embeddable is present, then both the panel title(s) and the relevant error message(s) will be printed as part of the error that is thrown. With these changes, if the attached flaky test fails again, we should at least have some more information on **why** it failed. ### Flaky Test Runner This was specifically only running the flaky `save_search_session_relative_time.ts` test suite: <a href="https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2024"><img src="https://user-images.githubusercontent.com/8698078/227240095-a336b03b-6499-4d80-8b60-5e325d54a4ae.png"/></a> ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

…153457) Closes elastic#97701 ## Summary This PR replaces all instances of `await testSubjects.missingOrFail('embeddableError');` with a call to the new `noErrorEmbeddablesPresent()` method from the `dashboardExpect` service. This `noErrorEmbeddablesPresent()` method adds some extra debugging logic to the error embeddable check where, if at least one error embeddable is present, then both the panel title(s) and the relevant error message(s) will be printed as part of the error that is thrown. With these changes, if the attached flaky test fails again, we should at least have some more information on **why** it failed. ### Flaky Test Runner This was specifically only running the flaky `save_search_session_relative_time.ts` test suite: <a href="https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2024"><img src="https://user-images.githubusercontent.com/8698078/227240095-a336b03b-6499-4d80-8b60-5e325d54a4ae.png"/></a> ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

kibanamachine · 2024-05-09T16:44:49Z

New failure: kibana-on-merge - main

kibanamachine · 2024-06-06T14:14:46Z

New failure: kibana-7-dot-17-es-8-dot-13-forward-compatibility - 7.17

kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Apr 20, 2021

botelastic bot added the needs-team Issues missing a team label label Apr 20, 2021

tylersmalley added Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas and removed needs-team Issues missing a team label labels Apr 20, 2021

Dosant added the Team:AppServices label Jul 19, 2021

This was referenced Jul 19, 2021

[wip] Unskip search session relative time test #106071

Closed

Lens embeddable doesn't set proper attributes when no results found #106085

Closed

wylieconlon mentioned this issue Jul 19, 2021

[Lens] Add render complete tags to empty states #106163

Merged

1 task

wylieconlon closed this as completed in #106163 Jul 22, 2021

kibanamachine reopened this Feb 14, 2023

mistic closed this as completed Feb 14, 2023

kibanamachine removed v8.0.0 v7.15.0 skipped-test blocker labels Feb 14, 2023

kibanamachine reopened this Mar 20, 2023

Heenawter added the impact:critical This issue should be addressed immediately due to a critical level of impact on the product. label Mar 20, 2023

Heenawter self-assigned this Mar 21, 2023

Heenawter mentioned this issue Mar 22, 2023

[Dashboard] Add better debugging to error embeddable checks #153457

Merged

2 tasks

Heenawter closed this as completed in #153457 Mar 24, 2023

kibanamachine reopened this May 9, 2024

Failing test: X-Pack Search Sessions Integration.x-pack/test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time·ts - Dashboard save a search sessions with relative time Saves and restores a session with relative time ranges #97701

Failing test: X-Pack Search Sessions Integration.x-pack/test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time·ts - Dashboard save a search sessions with relative time Saves and restores a session with relative time ranges #97701

Comments

kibanamachine commented Apr 20, 2021 • edited

elasticmachine commented Apr 20, 2021

tylersmalley commented Apr 20, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

kibanamachine commented Jul 18, 2021

elasticmachine commented Jul 19, 2021

Dosant commented Jul 19, 2021 • edited

wylieconlon commented Jul 19, 2021

Dosant commented Jul 20, 2021

kibanamachine commented Feb 14, 2023

kibanamachine commented Mar 20, 2023

ThomThomson commented Mar 21, 2023 • edited

kibanamachine commented May 9, 2024

kibanamachine commented Jun 6, 2024

kibanamachine commented Apr 20, 2021 •

edited

Dosant commented Jul 19, 2021 •

edited

ThomThomson commented Mar 21, 2023 •

edited