Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/connectors·ts - Actions and Triggers app Connectors should test a connector and display a successful result #88796

kibanamachine · 2021-01-20T06:20:49Z

A test failed on a tracked branch

Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
Wait timed out after 10026ms
    at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at onFailure (/dev/shm/workspace/parallel/16/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/dev/shm/workspace/parallel/16/kibana/test/common/services/retry/retry_for_success.ts:57:13)
    at Retry.try (/dev/shm/workspace/parallel/16/kibana/test/common/services/retry/retry.ts:32:14)
    at Context.<anonymous> (test/functional_with_es_ssl/apps/triggers_actions_ui/connectors.ts:122:7)
    at Object.apply (/dev/shm/workspace/parallel/16/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)

First failure: Jenkins Build

The text was updated successfully, but these errors were encountered:

kibanamachine · 2021-01-20T09:32:47Z

New failure: Jenkins Build

gmmorris · 2021-01-20T09:42:05Z

Here is the error I saw in the build log:

[00:00:53]             └-> should test a connector and display a successful result
[00:00:53]               └-> "before each" hook: global before each
[00:00:53]               │ debg TestSubjects.find(createActionButton)
[00:00:53]               │ debg Find.findByCssSelector('[data-test-subj="createActionButton"]') with timeout=10000
[00:00:53]               │ debg TestSubjects.click(.index-card)
[00:00:53]               │ debg Find.clickByCssSelector('[data-test-subj=".index-card"]') with timeout=10000
[00:00:53]               │ debg Find.findByCssSelector('[data-test-subj=".index-card"]') with timeout=10000
[00:00:54]               │ debg TestSubjects.setValue(nameInput, 0a58e2dffed345f282e8d89f403e8b57)
[00:00:54]               │ debg TestSubjects.click(nameInput)
[00:00:54]               │ debg Find.clickByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:00:54]               │ debg Find.findByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:00:54]               │ debg comboBox.set, comboBoxSelector: connectorIndexesComboBox
[00:00:54]               │ debg TestSubjects.find(connectorIndexesComboBox)
[00:00:54]               │ debg Find.findByCssSelector('[data-test-subj="connectorIndexesComboBox"]') with timeout=10000
[00:00:54]               │ debg comboBox.setElement, value: 1c3e9bcc1b80420cb70d515d8e2a289e
[00:00:54]               │ debg comboBox.isOptionSelected, value: 1c3e9bcc1b80420cb70d515d8e2a289e
[00:00:57]               │ debg TestSubjects.exists(~comboBoxOptionsList)
[00:00:57]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj~="comboBoxOptionsList"]') with timeout=2500
[00:00:57]               │ debg Find.allByCssSelector('.euiFilterSelectItem[title^="1c3e9bcc1b80420cb70d515d8e2a289e"]') with timeout=2500
[00:01:00]               │ debg Find.findByCssSelector('.euiFilterSelectItem') with timeout=5000
[00:01:00]               │ debg TestSubjects.exists(~comboBoxOptionsList)
[00:01:00]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj~="comboBoxOptionsList"]') with timeout=2500
[00:01:02]               │ debg --- retry.tryForTime error: [data-test-subj~="comboBoxOptionsList"] is not displayed
[00:01:03]               │ debg Find.clickByCssSelector('[data-test-subj="saveNewActionButton"]:not(disabled)') with timeout=10000
[00:01:03]               │ debg Find.findByCssSelector('[data-test-subj="saveNewActionButton"]:not(disabled)') with timeout=10000
[00:01:03]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:01:04]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:01:04]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:01:04]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:01:04]               │ debg Find.findByCssSelector('[data-test-subj="actionsList"] .euiFieldSearch') with timeout=10000
[00:01:04]               │ debg Find.findByCssSelector('.euiBasicTable[data-test-subj="actionsTable"]:not(.euiBasicTable-loading)') with timeout=10000
[00:01:04]               │ debg Find.findByCssSelector('[data-test-subj="actionsList"] table') with timeout=10000
[00:01:04]               │ debg Find.clickByCssSelector('[data-test-subj="connectorsTableCell-name"] button') with timeout=10000
[00:01:04]               │ debg Find.findByCssSelector('[data-test-subj="connectorsTableCell-name"] button') with timeout=10000
[00:01:05]               │ debg Find.clickByCssSelector('[data-test-subj="testConnectorTab"]') with timeout=10000
[00:01:05]               │ debg Find.findByCssSelector('[data-test-subj="testConnectorTab"]') with timeout=10000
[00:01:05]               │ debg TestSubjects.setValue(documentsJsonEditor, { "key": "value" })
[00:01:05]               │ debg TestSubjects.click(documentsJsonEditor)
[00:01:05]               │ debg Find.clickByCssSelector('[data-test-subj="documentsJsonEditor"]') with timeout=10000
[00:01:05]               │ debg Find.findByCssSelector('[data-test-subj="documentsJsonEditor"]') with timeout=10000
[00:01:05]               │ debg Find.clickByCssSelector('[data-test-subj="executeActionButton"]:not(disabled)') with timeout=10000
[00:01:05]               │ debg Find.findByCssSelector('[data-test-subj="executeActionButton"]:not(disabled)') with timeout=10000
[00:01:06]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:01:06]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:01:06]               │ proc [kibana]   log   [03:20:16.075] [error][actions][actions][plugins] error indexing documents: mapping set to strict, dynamic introduction of [key] within [_doc] is not allowed
[00:01:06]               │ proc [kibana]   log   [03:20:16.076] [warning][actions][actions][plugins] action execution failure: .index:68e67a70-5ace-11eb-8933-532fab7e1ae5: 0a58e2dffed345f282e8d89f403e8b57: error indexing documents: mapping set to strict, dynamic introduction of [key] within [_doc] is not allowed
[00:01:16]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:01:16]               │      Wait timed out after 10044ms
[00:01:16]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:01:16]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:01:26]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:01:26]               │      Wait timed out after 10064ms
[00:01:27]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:01:27]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:01:37]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:01:37]               │      Wait timed out after 10009ms
[00:01:37]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:01:37]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:01:47]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:01:47]               │      Wait timed out after 10051ms
[00:01:48]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:01:48]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:01:58]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:01:58]               │      Wait timed out after 10007ms
[00:01:58]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:01:58]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:02:08]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:02:08]               │      Wait timed out after 10010ms
[00:02:09]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:02:09]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:02:19]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:02:19]               │      Wait timed out after 10024ms
[00:02:19]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:02:19]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:02:29]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:02:29]               │      Wait timed out after 10060ms
[00:02:30]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:02:30]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:02:40]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:02:40]               │      Wait timed out after 10061ms
[00:02:40]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:02:40]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:02:51]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:02:51]               │      Wait timed out after 10049ms
[00:02:51]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:02:51]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:03:01]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:03:01]               │      Wait timed out after 10031ms
[00:03:02]               │ debg TestSubjects.find(executionSuccessfulResult)
[00:03:02]               │ debg Find.findByCssSelector('[data-test-subj="executionSuccessfulResult"]') with timeout=10000
[00:03:12]               │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:03:12]               │      Wait timed out after 10060ms
[00:03:12]               │ info Taking screenshot "/dev/shm/workspace/parallel/13/kibana/x-pack/test/functional/screenshots/failure/Actions and Triggers app Connectors should test a connector and display a successful result.png"
[00:03:12]               │ info Current URL is: http://localhost:61131/app/management/insightsAndAlerting/triggersActions/connectors
[00:03:12]               │ info Saving page source to: /dev/shm/workspace/parallel/13/kibana/x-pack/test/functional/failure_debug/html/Actions and Triggers app Connectors should test a connector and display a successful result.html
[00:03:12]               └- ✖ fail: Actions and Triggers app Connectors should test a connector and display a successful result
[00:03:12]               │      Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="executionSuccessfulResult"])
[00:03:12]               │ Wait timed out after 10060ms
[00:03:12]               │     at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:03:12]               │     at runMicrotasks (<anonymous>)
[00:03:12]               │     at processTicksAndRejections (internal/process/task_queues.js:93:5)
[00:03:12]               │       at onFailure (/dev/shm/workspace/parallel/13/kibana/test/common/services/retry/retry_for_success.ts:17:9)
[00:03:12]               │       at retryForSuccess (/dev/shm/workspace/parallel/13/kibana/test/common/services/retry/retry_for_success.ts:57:13)
[00:03:12]               │       at Retry.try (/dev/shm/workspace/parallel/13/kibana/test/common/services/retry/retry.ts:32:14)
[00:03:12]               │       at Context.<anonymous> (test/functional_with_es_ssl/apps/triggers_actions_ui/connectors.ts:122:7)
[00:03:12]               │       at Object.apply (/dev/shm/workspace/parallel/13/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)
[00:03:12]               │ 
[00:03:12]               │

Looks like it failed because the action failed (and the test is for the UI when the action. succeeds:

 [03:20:16.076] [warning][actions][actions][plugins] action execution failure: .index:68e67a70-5ace-11eb-8933-532fab7e1ae5: 0a58e2dffed345f282e8d89f403e8b57: error indexing documents: mapping set to strict, dynamic introduction of [key] within [_doc] is not allowed

kibanamachine · 2021-01-20T10:22:21Z

New failure: Jenkins Build

elasticmachine · 2021-01-20T13:25:30Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

mistic · 2021-01-20T13:32:27Z

The test keeps failing and looks flaky. Skipping for now.

master/8.0: 55d4331
7.x/7.12: 2a1adac
7.11/7.11.1: efd7b70

kibanamachine · 2021-01-20T14:20:24Z

New failure: Jenkins Build

kibanamachine · 2021-01-20T14:20:25Z

New failure: Jenkins Build

kibanamachine · 2021-01-20T15:09:31Z

New failure: Jenkins Build

kibanamachine · 2021-01-20T15:19:50Z

New failure: Jenkins Build

pmuellr · 2021-01-20T15:58:22Z

Huh, did ES change so that indices must have mappings now? If so, we should probably set the mappings when we create the connector, since it appears the index is always going to be a new index, given the connector generated (connectorName and indexName are UUID-based):

kibana/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/connectors.ts

Lines 281 to 292 in 7297cc3

    
           async function createIndexConnector(connectorName: string, indexName: string) { 
        
             await pageObjects.triggersActionsUI.clickCreateConnectorButton(); 
        
             await testSubjects.click('.index-card'); 
        
             await testSubjects.setValue('nameInput', connectorName); 
        
             await comboBox.set('connectorIndexesComboBox', indexName); 
        
             await find.clickByCssSelector('[data-test-subj="saveNewActionButton"]:not(disabled)'); 
        
             await pageObjects.common.closeToast(); 
        
           }

gmmorris · 2021-01-20T17:31:18Z

Huh, did ES change so that indices must have mappings now? If so, we should probably set the mappings when we create the connector, since it appears the index is always going to be a new index, given the connector generated (connectorName and indexName are UUID-based):

Nope. I managed to reproduce locally - the UI driver sometime selects the wrong index in the ComboBox.
When this happens it defaults to the first in the list, which is the system indices starting with ., such as .kibana_task_manager, at which point it fails to index the document.

kibanamachine · 2021-11-06T03:26:43Z

New failure: CI Build - 7.16

kibanamachine · 2021-11-26T12:08:21Z

New failure: CI Build - 8.0

kibanamachine · 2021-12-17T04:52:08Z

New failure: CI Build - 7.17

kibanamachine · 2021-12-21T07:03:31Z

New failure: CI Build - main

kibanamachine · 2021-12-26T22:04:29Z

New failure: CI Build - 8.0

kibanamachine · 2022-01-14T11:00:46Z

New failure: CI Build - main

mistic · 2022-01-17T13:58:23Z

This test keeps failing and looks flaky. Skipping for now.

main/8.1: c4b4cf1
8.0: 5a38f03

kibanamachine · 2022-02-01T12:52:49Z

New failure: CI Build - 7.17

kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Jan 20, 2021

mistic added the Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) label Jan 20, 2021

mikecote added this to To-Do (Ordered by priority) in Make it Action Jan 20, 2021

mistic added a commit that referenced this issue Jan 20, 2021

skip flaky suite (#88796)

55d4331

mistic added a commit that referenced this issue Jan 20, 2021

skip flaky suite (#88796)

2a1adac

mistic added blocker skipped-test v8.0.0 labels Jan 20, 2021

gmmorris self-assigned this Jan 20, 2021

gmmorris moved this from To-Do (Ordered by priority) to In Progress in Make it Action Jan 20, 2021

mistic added a commit that referenced this issue Jan 20, 2021

skip flaky suite (#88796)

efd7b70

gmmorris mentioned this issue Jan 20, 2021

Ensures we always select the correct index in UI tests #88876

Merged

9 tasks

gmmorris moved this from In Progress to In Review in Make it Action Jan 20, 2021

gmmorris closed this as completed in #88876 Jan 21, 2021

Make it Action automation moved this from In Review to Done (Ordered by most recent) Jan 21, 2021

kibanamachine reopened this Nov 6, 2021

gmmorris added the Feature:Alerting/RulesManagement Issues related to the Rules Management UX label Nov 8, 2021

mikecote unassigned gmmorris Nov 8, 2021

gmmorris added this to To-Do (Ordered by priority) in Kibana Alerting Nov 10, 2021

mikecote mentioned this issue Nov 15, 2021

Fix flaky tests potentially caused by transition to buildkite #118535

Closed

mikecote removed this from To-Do (Ordered by priority) in Kibana Alerting Nov 15, 2021

mikecote removed skipped-test blocker labels Dec 14, 2021

mikecote added this to Backlog in Kibana Alerting Dec 14, 2021

XavierM removed this from Backlog in Kibana Alerting Jan 6, 2022

mistic added a commit that referenced this issue Jan 17, 2022

skip flaky suite (#88796)

c4b4cf1

mistic added a commit that referenced this issue Jan 17, 2022

skip flaky suite (#88796)

5a38f03

mistic added blocker skipped-test v8.1.0 labels Jan 17, 2022

ogupte pushed a commit to ogupte/kibana that referenced this issue Jan 28, 2022

skip flaky suite (elastic#88796)

945ccf9

kobelb added the needs-team Issues missing a team label label Jan 31, 2022

botelastic bot removed the needs-team Issues missing a team label label Jan 31, 2022

doakalexi self-assigned this Sep 9, 2022

doakalexi mentioned this issue Sep 19, 2022

[ResponseOps][Connectors] Fixing flaky connectors·ts test #140989

Merged

1 task

doakalexi closed this as completed in #140989 Sep 22, 2022

kibanamachine removed v8.1.0 v8.0.0 skipped-test blocker labels Sep 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/connectors·ts - Actions and Triggers app Connectors should test a connector and display a successful result #88796

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/connectors·ts - Actions and Triggers app Connectors should test a connector and display a successful result #88796

kibanamachine commented Jan 20, 2021 •

edited

kibanamachine commented Jan 20, 2021

gmmorris commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

elasticmachine commented Jan 20, 2021

mistic commented Jan 20, 2021 •

edited

kibanamachine commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

pmuellr commented Jan 20, 2021

gmmorris commented Jan 20, 2021

kibanamachine commented Nov 6, 2021

kibanamachine commented Nov 26, 2021

kibanamachine commented Dec 17, 2021

kibanamachine commented Dec 21, 2021

kibanamachine commented Dec 26, 2021

kibanamachine commented Jan 14, 2022

mistic commented Jan 17, 2022

kibanamachine commented Feb 1, 2022

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/connectors·ts - Actions and Triggers app Connectors should test a connector and display a successful result #88796

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/connectors·ts - Actions and Triggers app Connectors should test a connector and display a successful result #88796

Comments

kibanamachine commented Jan 20, 2021 • edited

kibanamachine commented Jan 20, 2021

gmmorris commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

elasticmachine commented Jan 20, 2021

mistic commented Jan 20, 2021 • edited

kibanamachine commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

kibanamachine commented Jan 20, 2021

pmuellr commented Jan 20, 2021

gmmorris commented Jan 20, 2021

kibanamachine commented Nov 6, 2021

kibanamachine commented Nov 26, 2021

kibanamachine commented Dec 17, 2021

kibanamachine commented Dec 21, 2021

kibanamachine commented Dec 26, 2021

kibanamachine commented Jan 14, 2022

mistic commented Jan 17, 2022

kibanamachine commented Feb 1, 2022

kibanamachine commented Jan 20, 2021 •

edited

mistic commented Jan 20, 2021 •

edited