[Security Solution][Detections] Implements indicator match rule cypress test #84323

MadameSheema · 2020-11-25T12:14:53Z

Summary

In this PR we are adding a new test in order to check that an Indicator Match rule can be created and is able to populate alerts.

For this test we have created two different archives:

threat_data
threat indicator

The threat_data archive contains an auditbeat type document that matches with threat-data-* index pattern.

The threat_index archive, contains two documents that matches with threat-index-* index pattern that contains the threats, that matches with the threat-data.

The test:

Creates an Indicator Match rule using the UI
Checks that the created rule has all the expected data
Checks that the rule generates the expected alert and has the expected values.

MadameSheema · 2020-11-30T07:55:53Z

@elasticmachine merge upstream

FrankHassanabad · 2020-11-30T18:53:43Z

Just a suggestion but es_archiver does support non gzip of data.

Looking at:

x-pack/test/security_solution_cypress/es_archives/threat_data/data.json.gz

That looks like it only has a single record within it, so I would just not gzip it up and check it in as data.json and then for the mapping.json here and that data.json:

 ~/projects/kibana/x-pack/test/security_solution_cypress/es_archives/threat_data/mappings.json

I would trim the mappings down if possible to only match the parts of the data set you need to match. Technically everything will work out ok if we use ECS compatible mappings even if they don't have the full data set including the full mapping. Also nothing wrong with adding something to the mappings that don't exist in ECS as long as we don't use a conflicting field name from ECS.

For example, on backend e2e tests I am using very small data sets and small mappings like these recent ones:
https://github.com/elastic/kibana/tree/master/x-pack/test/functional/es_archives/rule_exceptions

The only "required field" is @timestamp and really because ECS is a constant changing/evolving standard it was just easier for me to conduct tests that way. For example, soon ECS is going to change a lot of mappings to be wildcard:
https://github.com/elastic/ecs/releases

But in my above tests it shouldn't matter since I only use a @timestamp and a made-up non conflicting field. I didn't gzip anything up as the record contents were very small and so was the mappings. From a maintenance perspective I have been noticing more people with newer e2e tests on the backend such as the alerting team doing these patterns of not zipping things and using small data sets so I think it's catching on as being easier.

Optional here, do what's best, just mentioning this and depends on what you're testing. I typically these days try to keep the least amount of mapping/data for functional tests since ECS is constantly evolving. For other tests or issues with backwards compatibility with ECS I try to do the same thing and just concentrate on types such as keyword/wildcard, etc... and not directly test something large or the actual fields of ECS if I don't have to.

Although I might need to do more of those later if we have bugs directly related to different ECS mapping types. However, even then I try to abstract it away into something that doesn't directly involve ECS mappings if I can (other than @timestamp needing to be there).

That also usually catches bugs if someone is relying a field to exist from ECS as an added bonus since all those fields should be optional other than @tiemstamp

FrankHassanabad · 2020-11-30T20:43:29Z

x-pack/test/security_solution_cypress/es_archives/threat_data/mappings.json

+  "type": "index",
+  "value": {
+    "aliases": {
+      "playing": {


Did you want this here where the alias is playing?

I missed that, thanks @FrankHassanabad :)

achuguy · 2020-12-01T16:02:07Z

x-pack/plugins/security_solution/cypress/tasks/create_new_rule.ts

-          return alertCount > 0;
-        });
-    },
-    { interval: 500, timeout: 12000 }


Was there a reason for this to be here previously, i.e. timing reasons? Are other tests relying on this?

The method is on the same file but displayed in a different order. I just sorted the methods alphabetically.

I didn't see parameters for interval and timeout in the other method. Were the values here used at all?

MadameSheema · 2020-12-02T11:04:26Z

@FrankHassanabad I think I have simplified and fix the data:

I have fixed the alias name of the threat_data mapping
I have simplified the threat_index data and mapping, using just one doc and deleting all the fields we don't use for the test
I have change the threat_index to use a non ECS field to check the rule is able to work with this type of fields.

Can you please review the PR again, lots of thanks :)

MadameSheema · 2020-12-02T11:04:43Z

@elasticmachine merge upstream

… indicator-match

FrankHassanabad · 2020-12-02T15:49:06Z

x-pack/plugins/security_solution/cypress/tasks/create_new_rule.ts

+        const alertCount = parseInt(countText, 10) || 0;
+        return alertCount > 0;
+      });
+  });


Nice!

For the backend e2e, I started just like this, but then later I had to add a specific number to wait for a specific number of alerts as even with things like refresh=wait_for within the code there does exist some time before the Nth alert shows up.

I had some flakiness where 1 signal would show up on the backend but I needed to wait for like 2 or 3. No changes requested, but if we see flaky we will probably have to add that integer here.

FrankHassanabad

LGTM! Thanks for the tests here for exceptions. Users use exceptions A LOT and we are fixing bugs/refactoring parts of it so this is going to be so great for us to ensure that the exceptions are all operating as expected.

kibanamachine · 2020-12-02T19:47:41Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: 0d09335
Flaky suites:
- xpack-kibana-ciGroup10

Test Failures

Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/alert_create_flyout·ts.Actions and Triggers app create alert should show save confirmation before creating alert with no actions

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: Actions and Triggers app
[00:00:00]           └-> "before all" hook
[00:03:49]           └-: create alert
[00:03:49]             └-> "before all" hook
[00:03:49]             └-> "before all" hook
[00:03:49]               │ debg navigating to triggersActions url: http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:49]               │ debg navigate to: http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:50]               │ debg browser[INFO] http://localhost:61111/app/management/insightsAndAlerting/triggersActions?_t=1606934658425 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:03:50]               │
[00:03:50]               │ debg browser[INFO] http://localhost:61111/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:03:50]               │ debg ... sleep(700) start
[00:03:50]               │ debg ... sleep(700) end
[00:03:50]               │ debg returned from get, calling refresh
[00:03:51]               │ debg browser[INFO] http://localhost:61111/app/management/insightsAndAlerting/triggersActions?_t=1606934658425 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:03:51]               │
[00:03:51]               │ debg browser[INFO] http://localhost:61111/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:03:51]               │ debg currentUrl = http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:51]               │          appUrl = http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:51]               │ debg TestSubjects.find(kibanaChrome)
[00:03:51]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:03:53]               │ debg ... sleep(501) start
[00:03:53]               │ debg ... sleep(501) end
[00:03:53]               │ debg in navigateTo url = http://localhost:61111/app/management/insightsAndAlerting/triggersActions/alerts
[00:03:53]               │ debg --- retry.try error: URL changed, waiting for it to settle
[00:03:54]               │ debg ... sleep(501) start
[00:03:54]               │ debg ... sleep(501) end
[00:03:54]               │ debg in navigateTo url = http://localhost:61111/app/management/insightsAndAlerting/triggersActions/alerts
[00:03:54]               │ debg TestSubjects.exists(statusPageContainer)
[00:03:54]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:03:57]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:03:57]               │ debg TestSubjects.click(alertsTab)
[00:03:57]               │ debg Find.clickByCssSelector('[data-test-subj="alertsTab"]') with timeout=10000
[00:03:57]               │ debg Find.findByCssSelector('[data-test-subj="alertsTab"]') with timeout=10000
[00:03:57]             └-> should create an alert
[00:03:57]               └-> "before each" hook: global before each
[00:03:57]               │ debg Find.findByCssSelector('[data-test-subj="createAlertButton"],[data-test-subj="createFirstAlertButton"]') with timeout=10000
[00:03:58]               │ debg TestSubjects.setValue(alertNameInput, 0ee1f45f829543af80922e1c6ebdf48f)
[00:03:58]               │ debg TestSubjects.click(alertNameInput)
[00:03:58]               │ debg Find.clickByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:03:58]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:03:59]               │ debg TestSubjects.click(.index-threshold-SelectOption)
[00:03:59]               │ debg Find.clickByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:03:59]               │ debg Find.findByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:03:59]               │ debg TestSubjects.click(selectIndexExpression)
[00:03:59]               │ debg Find.clickByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:03:59]               │ debg Find.findByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:03:59]               │ debg Find.findByCssSelector('#indexSelectSearchBox') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('.euiFilterSelectItem') with timeout=10000
[00:04:00]               │ debg TestSubjects.click(thresholdAlertTimeFieldSelect)
[00:04:00]               │ debg Find.clickByCssSelector('[data-test-subj="thresholdAlertTimeFieldSelect"]') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj="thresholdAlertTimeFieldSelect"]') with timeout=10000
[00:04:00]               │ debg Find.allByCssSelector('#thresholdTimeField option') with timeout=10000
[00:04:00]               │ debg TestSubjects.click(closePopover)
[00:04:00]               │ debg Find.clickByCssSelector('[data-test-subj="closePopover"]') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj="closePopover"]') with timeout=10000
[00:04:00]               │ debg TestSubjects.find(alertNameInput)
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:00]               │ debg TestSubjects.click(.slack-ActionTypeSelectOption)
[00:04:00]               │ debg Find.clickByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:01]               │ debg TestSubjects.click(addNewActionConnectorButton-.slack)
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:01]               │ debg TestSubjects.setValue(nameInput, 15b0e087906b4c1f8ea1a01c767057e7)
[00:04:01]               │ debg TestSubjects.click(nameInput)
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:01]               │ debg TestSubjects.setValue(slackWebhookUrlInput, https://test)
[00:04:01]               │ debg TestSubjects.click(slackWebhookUrlInput)
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:02]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:02]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:02]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:02]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:02]               │ debg TestSubjects.setValue(messageTextArea, test message )
[00:04:02]               │ debg TestSubjects.click(messageTextArea)
[00:04:02]               │ debg Find.clickByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:02]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:03]               │ debg TestSubjects.click(messageAddVariableButton)
[00:04:03]               │ debg Find.clickByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:03]               │ debg Find.findByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:03]               │ debg TestSubjects.click(variableMenuButton-0)
[00:04:03]               │ debg Find.clickByCssSelector('[data-test-subj="variableMenuButton-0"]') with timeout=10000
[00:04:03]               │ debg Find.findByCssSelector('[data-test-subj="variableMenuButton-0"]') with timeout=10000
[00:04:05]               │ debg TestSubjects.click(messageAddVariableButton)
[00:04:05]               │ debg Find.clickByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:05]               │ debg TestSubjects.click(variableMenuButton-1)
[00:04:05]               │ debg Find.clickByCssSelector('[data-test-subj="variableMenuButton-1"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('[data-test-subj="variableMenuButton-1"]') with timeout=10000
[00:04:05]               │ debg TestSubjects.click(saveAlertButton)
[00:04:05]               │ debg Find.clickByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:07]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:07]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:07]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:07]               │ debg TestSubjects.find(alertSearchField)
[00:04:07]               │ debg Find.findByCssSelector('[data-test-subj="alertSearchField"]') with timeout=10000
[00:04:08]               │ debg Find.findByCssSelector('.euiBasicTable[data-test-subj="alertsList"]:not(.euiBasicTable-loading)') with timeout=10000
[00:04:08]               │ debg Find.findByCssSelector('[data-test-subj="alertsList"] table') with timeout=10000
[00:04:08]               └- ✓ pass  (11.0s) "Actions and Triggers app create alert should create an alert"
[00:04:08]             └-> should create an alert with actions in multiple groups
[00:04:08]               └-> "before each" hook: global before each
[00:04:08]               │ debg Find.findByCssSelector('[data-test-subj="createAlertButton"],[data-test-subj="createFirstAlertButton"]') with timeout=10000
[00:04:08]               │ debg TestSubjects.setValue(alertNameInput, 4cd043b8806e44c3b33b86488ff22afd)
[00:04:08]               │ debg TestSubjects.click(alertNameInput)
[00:04:08]               │ debg Find.clickByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:08]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:09]               │ debg TestSubjects.click(test.always-firing-SelectOption)
[00:04:09]               │ debg Find.clickByCssSelector('[data-test-subj="test.always-firing-SelectOption"]') with timeout=10000
[00:04:09]               │ debg Find.findByCssSelector('[data-test-subj="test.always-firing-SelectOption"]') with timeout=10000
[00:04:09]               │ debg TestSubjects.click(.slack-ActionTypeSelectOption)
[00:04:09]               │ debg Find.clickByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:09]               │ debg Find.findByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:10]               │ debg TestSubjects.click(addNewActionConnectorButton-.slack)
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:10]               │ debg TestSubjects.setValue(nameInput, f3381162f3b84fd0bfccd5a6192451aa)
[00:04:10]               │ debg TestSubjects.click(nameInput)
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:10]               │ debg TestSubjects.setValue(slackWebhookUrlInput, https://test)
[00:04:10]               │ debg TestSubjects.click(slackWebhookUrlInput)
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:11]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:11]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:11]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:11]               │ debg TestSubjects.setValue(messageTextArea, test message )
[00:04:11]               │ debg TestSubjects.click(messageTextArea)
[00:04:11]               │ debg Find.clickByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:11]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj="alertActionAccordion-0"] [data-test-subj="messageTextArea"]') with timeout=10000
[00:04:12]               │ debg TestSubjects.click(addAlertActionButton)
[00:04:12]               │ debg Find.clickByCssSelector('[data-test-subj="addAlertActionButton"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj="addAlertActionButton"]') with timeout=10000
[00:04:12]               │ debg TestSubjects.click(.slack-ActionTypeSelectOption)
[00:04:12]               │ debg Find.clickByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:12]               │ debg TestSubjects.setValue(messageTextArea, test message )
[00:04:12]               │ debg TestSubjects.click(messageTextArea)
[00:04:12]               │ debg Find.clickByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:13]               │ debg Find.findByCssSelector('[data-test-subj="alertActionAccordion-1"] [data-test-subj="messageTextArea"]') with timeout=10000
[00:04:14]               │ debg TestSubjects.click(addNewActionConnectorActionGroup-1)
[00:04:14]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1"]') with timeout=10000
[00:04:14]               │ debg TestSubjects.click(addNewActionConnectorActionGroup-1-option-other)
[00:04:14]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1-option-other"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1-option-other"]') with timeout=10000
[00:04:14]               │ debg TestSubjects.click(saveAlertButton)
[00:04:14]               │ debg Find.clickByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:16]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:16]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:16]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:16]               │ debg TestSubjects.find(alertSearchField)
[00:04:16]               │ debg Find.findByCssSelector('[data-test-subj="alertSearchField"]') with timeout=10000
[00:04:17]               │ debg Find.findByCssSelector('.euiBasicTable[data-test-subj="alertsList"]:not(.euiBasicTable-loading)') with timeout=10000
[00:04:17]               │ debg Find.findByCssSelector('[data-test-subj="alertsList"] table') with timeout=10000
[00:04:17]               └- ✓ pass  (8.8s) "Actions and Triggers app create alert should create an alert with actions in multiple groups"
[00:04:17]             └-> should show save confirmation before creating alert with no actions
[00:04:17]               └-> "before each" hook: global before each
[00:04:17]               │ debg Find.findByCssSelector('[data-test-subj="createAlertButton"],[data-test-subj="createFirstAlertButton"]') with timeout=10000
[00:04:17]               │ debg TestSubjects.setValue(alertNameInput, 47c5ede154514ace8509c2f552927c6f)
[00:04:17]               │ debg TestSubjects.click(alertNameInput)
[00:04:17]               │ debg Find.clickByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:17]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:18]               │ debg TestSubjects.click(.index-threshold-SelectOption)
[00:04:18]               │ debg Find.clickByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:04:18]               │ debg Find.findByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:04:18]               │ debg TestSubjects.click(selectIndexExpression)
[00:04:18]               │ debg Find.clickByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:04:18]               │ debg Find.findByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:04:18]               │ debg Find.findByCssSelector('#indexSelectSearchBox') with timeout=10000
[00:04:28]               │ info Taking screenshot "/dev/shm/workspace/parallel/11/kibana/x-pack/test/functional/screenshots/failure/Actions and Triggers app create alert should show save confirmation before creating alert with no actions.png"
[00:04:29]               │ info Current URL is: http://localhost:61111/app/management/insightsAndAlerting/triggersActions/alerts
[00:04:29]               │ info Saving page source to: /dev/shm/workspace/parallel/11/kibana/x-pack/test/functional/failure_debug/html/Actions and Triggers app create alert should show save confirmation before creating alert with no actions.html
[00:04:29]               └- ✖ fail: Actions and Triggers app create alert should show save confirmation before creating alert with no actions
[00:04:29]               │      TimeoutError: Waiting for element to be located By(css selector, #indexSelectSearchBox)
[00:04:29]               │ Wait timed out after 10020ms
[00:04:29]               │       at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:04:29]               │       at runMicrotasks (<anonymous>)
[00:04:29]               │       at processTicksAndRejections (internal/process/task_queues.js:97:5)
[00:04:29]               │ 
[00:04:29]               │

Stack Trace

TimeoutError: Waiting for element to be located By(css selector, #indexSelectSearchBox)
Wait timed out after 10020ms
    at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  remoteStacktrace: ''
}

Metrics [docs]

Distributable file count

id	before	after	diff
`default`	43202	43203	+1

History

💔 Build #91353 failed 4cb4fa9
💔 Build #91320 failed 3885bf1
💔 Build #90812 failed ebe9554
💔 Build #90786 failed 19a210d
💔 Build #90664 failed 4040d4e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…ss test (elastic#84323) * implemnts indicator match rule cypress test * fixes merge issue * fixes type check issues * fixes mapping * simplifies data * fixes excpetions flakiness * fixes alerts test Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

* master: (40 commits) fix: 🐛 don't add separator befor group on no main items (elastic#83166) [Security Solution][Detections] Implements indicator match rule cypress test (elastic#84323) [APM] Add APM agent config options (elastic#84678) Fixed a11y issue on rollup jobs table selection (elastic#84567) [Discover] Refactor getContextUrl to separate file (elastic#84503) [Embeddable] Export CSV action for Lens embeddables in dashboard (elastic#83654) [TSVB] [Cleanup] Remove extra dateFormat props (elastic#84749) [Lens] Migrate legacy es client and remove total hits as int (elastic#84340) Improve logging pipeline in @kbn/legacy-logging (elastic#84629) Catch @hapi/podium errors (elastic#84575) [Discover] Unskip date histogram test (elastic#84727) Rename server.xsrf.whitelist to server.xsrf.allowlist (elastic#84791) [Enterprise Search] Fix schema errors button (elastic#84842) [APM] Removes react-sticky dependency in favor of using CSS (elastic#84589) [Maps] Always initialize routes on server-startup (elastic#84806) [Fleet] EPM support to handle uploaded file paths (elastic#84708) [Snapshot Restore] Fix initial policy form state (elastic#83928) Upgrade Node.js to version 14 (elastic#83425) [Security Solution] Keep Endpoint policies up to date with license changes (elastic#83992) [Security Solution][Exceptions] Implement exceptions for ML rules (elastic#84006) ...

* master: (236 commits) fix: 🐛 don't add separator befor group on no main items (elastic#83166) [Security Solution][Detections] Implements indicator match rule cypress test (elastic#84323) [APM] Add APM agent config options (elastic#84678) Fixed a11y issue on rollup jobs table selection (elastic#84567) [Discover] Refactor getContextUrl to separate file (elastic#84503) [Embeddable] Export CSV action for Lens embeddables in dashboard (elastic#83654) [TSVB] [Cleanup] Remove extra dateFormat props (elastic#84749) [Lens] Migrate legacy es client and remove total hits as int (elastic#84340) Improve logging pipeline in @kbn/legacy-logging (elastic#84629) Catch @hapi/podium errors (elastic#84575) [Discover] Unskip date histogram test (elastic#84727) Rename server.xsrf.whitelist to server.xsrf.allowlist (elastic#84791) [Enterprise Search] Fix schema errors button (elastic#84842) [APM] Removes react-sticky dependency in favor of using CSS (elastic#84589) [Maps] Always initialize routes on server-startup (elastic#84806) [Fleet] EPM support to handle uploaded file paths (elastic#84708) [Snapshot Restore] Fix initial policy form state (elastic#83928) Upgrade Node.js to version 14 (elastic#83425) [Security Solution] Keep Endpoint policies up to date with license changes (elastic#83992) [Security Solution][Exceptions] Implement exceptions for ML rules (elastic#84006) ...

…overy-action-group * upstream/master: (48 commits) [Lens] accessibility screen reader issues (elastic#84395) [Logs UI] Fetch single log entries via a search strategy (elastic#81710) fix: 🐛 don't add separator befor group on no main items (elastic#83166) [Security Solution][Detections] Implements indicator match rule cypress test (elastic#84323) [APM] Add APM agent config options (elastic#84678) Fixed a11y issue on rollup jobs table selection (elastic#84567) [Discover] Refactor getContextUrl to separate file (elastic#84503) [Embeddable] Export CSV action for Lens embeddables in dashboard (elastic#83654) [TSVB] [Cleanup] Remove extra dateFormat props (elastic#84749) [Lens] Migrate legacy es client and remove total hits as int (elastic#84340) Improve logging pipeline in @kbn/legacy-logging (elastic#84629) Catch @hapi/podium errors (elastic#84575) [Discover] Unskip date histogram test (elastic#84727) Rename server.xsrf.whitelist to server.xsrf.allowlist (elastic#84791) [Enterprise Search] Fix schema errors button (elastic#84842) [APM] Removes react-sticky dependency in favor of using CSS (elastic#84589) [Maps] Always initialize routes on server-startup (elastic#84806) [Fleet] EPM support to handle uploaded file paths (elastic#84708) [Snapshot Restore] Fix initial policy form state (elastic#83928) Upgrade Node.js to version 14 (elastic#83425) ...

…ss test (#84323) (#84882) * implemnts indicator match rule cypress test * fixes merge issue * fixes type check issues * fixes mapping * simplifies data * fixes excpetions flakiness * fixes alerts test Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

implemnts indicator match rule cypress test

5a8ae16

MadameSheema self-assigned this Nov 25, 2020

MadameSheema added v7.11.0 Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detections and Resp Security Detection Response Team release_note:skip Skip the PR/issue when compiling release notes v8.0.0 labels Nov 25, 2020

MadameSheema marked this pull request as ready for review November 25, 2020 15:06

MadameSheema requested review from a team as code owners November 25, 2020 15:06

Merge branch 'master' into indicator-match

a93d599

MadameSheema requested review from FrankHassanabad, peluja1012 and spong November 30, 2020 12:21

MadameSheema added 3 commits November 30, 2020 13:21

Merge branch 'master' into indicator-match

4040d4e

fixes merge issue

19a210d

fixes type check issues

ebe9554

FrankHassanabad reviewed Nov 30, 2020

View reviewed changes

achuguy reviewed Dec 1, 2020

View reviewed changes

MadameSheema added 2 commits December 2, 2020 09:35

fixes mapping

a0e14b9

simplifies data

5b8f92e

kibanamachine and others added 3 commits December 2, 2020 06:04

Merge branch 'master' into indicator-match

3885bf1

fixes excpetions flakiness

afbfcd3

Merge branch 'indicator-match' of github.com:MadameSheema/kibana into…

4cb4fa9

… indicator-match

FrankHassanabad reviewed Dec 2, 2020

View reviewed changes

FrankHassanabad approved these changes Dec 2, 2020

View reviewed changes

fixes alerts test

0d09335

MadameSheema merged commit 37e9070 into elastic:master Dec 3, 2020

MadameSheema deleted the indicator-match branch December 3, 2020 10:48

MadameSheema mentioned this pull request Dec 3, 2020

[7.x] [Security Solution][Detections] Implements indicator match rule cypress test (#84323) #84882

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][Detections] Implements indicator match rule cypress test #84323

[Security Solution][Detections] Implements indicator match rule cypress test #84323

MadameSheema commented Nov 25, 2020 •

edited

MadameSheema commented Nov 30, 2020

FrankHassanabad commented Nov 30, 2020

FrankHassanabad Nov 30, 2020

MadameSheema Dec 2, 2020

achuguy Dec 1, 2020

MadameSheema Dec 2, 2020

achuguy Dec 2, 2020

MadameSheema commented Dec 2, 2020

MadameSheema commented Dec 2, 2020

FrankHassanabad Dec 2, 2020

FrankHassanabad left a comment

kibanamachine commented Dec 2, 2020

Standard Out

Stack Trace

[Security Solution][Detections] Implements indicator match rule cypress test #84323

[Security Solution][Detections] Implements indicator match rule cypress test #84323

Conversation

MadameSheema commented Nov 25, 2020 • edited

Summary

MadameSheema commented Nov 30, 2020

FrankHassanabad commented Nov 30, 2020

FrankHassanabad Nov 30, 2020

Choose a reason for hiding this comment

MadameSheema Dec 2, 2020

Choose a reason for hiding this comment

achuguy Dec 1, 2020

Choose a reason for hiding this comment

MadameSheema Dec 2, 2020

Choose a reason for hiding this comment

achuguy Dec 2, 2020

Choose a reason for hiding this comment

MadameSheema commented Dec 2, 2020

MadameSheema commented Dec 2, 2020

FrankHassanabad Dec 2, 2020

Choose a reason for hiding this comment

FrankHassanabad left a comment

Choose a reason for hiding this comment

kibanamachine commented Dec 2, 2020

💛 Build succeeded, but was flaky

Test Failures

Standard Out

Stack Trace

Metrics [docs]

Distributable file count

History

MadameSheema commented Nov 25, 2020 •

edited