[Security Solution][API testing] Move and restructures Rule execution logic #170765

WafaaNasr · 2023-11-07T16:11:49Z

Summary

Following the initial work in this #166755

Addresses part of [Security Solution] Restructure API integration tests into meaningful groups that run in parallel on CI #151902 for rule execution logic
Moved the utility files associated with rule execution logic to the new directory security_solution_api_integration. Files not actively used in the previous folder were moved, while duplicate files remained in their original positions.
Updated the CodeOwner file for the newly moved tests
Old/new group details, decisions, and execution time are mentioned in thisdocument
Added new Alert archive for version 8.8.0
Resolved the issue with the query.ts test where the execution logic is executed last, encompassing the "query" test because it was unloading the alerts document and led to failures in subsequent tests.
For Alert As Data in Serverless the alert ancestor will be a data-stream however in ESS will be .internal.alerts-security.alerts-default-000001'

Action	File	New Path if moved
Delete	security_and_spaces/rule_execution_logic	-
Delete	security_and_spaces/group5	-
Move	detection_engine_api_integration/security_and_spaces/group5	detections_response/default_license/rule_execution_logic/keyword_family
Move	detection_engine_api_integration/security_and_spaces/rule_execution_logic	detections_response/default_license/rule_execution_logic/execution_logic
Move	detection_engine_api_integration/security_and_spaces/group1/ignore_fields	detections_response/default_license/rule_execution_logic/ignore_fields.ts
Move	detection_engine_api_integration/security_and_spaces/group1/runtime	detections_response/default_license/rule_execution_logic/runtime.ts
Move	detection_engine_api_integration/security_and_spaces/group1/timestamps	detections_response/default_license/rule_execution_logic/timestamps.ts

…ructure-rule_execution_test

…fix'

…om/WafaaNasr/kibana into move-structure-rule_execution_test

…ructure-rule_execution_test

…ot unload the alerts data

…ructure-rule_execution_test

e40pud

Looks great, thank you for doing this! Left a few comments

e40pud · 2023-11-16T12:13:54Z

x-pack/test/functional/es_archives/security_solution/timestamp_override_6/mappings.json

@@ -20,7 +20,6 @@
    },
    "settings": {
      "index": {
-        "refresh_interval": "1s",


Just curious, why did we remove refresh_interval?

Sure! The refresh_interval is not a valid config in Serverless, and We found out that we don't even need for ESS either since out tests don't valid against that fields

e40pud · 2023-11-16T12:14:46Z

x-pack/test/security_solution_api_integration/config/serverless/config.base.ts

    return {
      ...svlSharedConfig.getAll(),
+      services: {


Isn't it the same as just services,?

Yes true, I added it in case we want to pass additional services for the Security

e40pud · 2023-11-16T12:16:47Z

.../test_suites/detections_response/default_license/rule_execution_logic/execution_logic/eql.ts

+  const config = getService('config');
+  const isServerless = config.get('serverless');
+  const dataPathBuilder = new EsArchivePathBuilder(isServerless);
+  const path = dataPathBuilder.getPath('auditbeat/hosts');


Could we rename path to auditbeatPath? I think it will be more descriptive this way.

e40pud · 2023-11-16T12:17:31Z

.../test_suites/detections_response/default_license/rule_execution_logic/execution_logic/eql.ts

-      expect(previewAlerts.length).eql(maxSignals * 3);
-      const shellSignals = previewAlerts.filter((alert) => alert._source?.[ALERT_DEPTH] === 2);
+      expect(previewAlerts.length).eql(maxAlerts * 3);
+      const shellalerts = previewAlerts.filter((alert) => alert._source?.[ALERT_DEPTH] === 2);


shellAlerts?

e40pud · 2023-11-16T12:18:28Z

...detections_response/default_license/rule_execution_logic/execution_logic/machine_learning.ts

+  const config = getService('config');
+  const isServerless = config.get('serverless');
+  const dataPathBuilder = new EsArchivePathBuilder(isServerless);
+  const path = dataPathBuilder.getPath('auditbeat/hosts');


also here, maybe auditbeatPath will be more descriptive

...detections_response/default_license/rule_execution_logic/execution_logic/machine_learning.ts

…om/WafaaNasr/kibana into move-structure-rule_execution_test

…ructure-rule_execution_test

kibana-ci · 2023-11-16T14:20:27Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: fde47c6

Failed CI Steps

Defend Workflows Cypress Tests on Serverless #1

Metrics [docs]

✅ unchanged

History

💛 Build #176422 was flaky 34ea580
💛 Build #176321 was flaky c48c7e4
💔 Build #176251 failed efb7210
💔 Build #176197 failed 11435f1
💚 Build #175808 succeeded 1526cd0
💔 Build #175656 failed fab68bd

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @WafaaNasr

jbudz

ftr_configs.yml

WafaaNasr added 2 commits November 7, 2023 17:09

add keyword family

bf7ef58

remove group5

cee4086

WafaaNasr added release_note:skip Skip the PR/issue when compiling release notes FTR labels Nov 7, 2023

WafaaNasr self-assigned this Nov 7, 2023

WafaaNasr and others added 25 commits November 8, 2023 15:24

fix linting

aed2dda

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

00fc693

…ructure-rule_execution_test

remove refresh_interval setting

4bd661a

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

5e10e0f

…ructure-rule_execution_test

fix linting

9421781

move rule_execution_logic

17fd9d1

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

5750a13

…fix'

move all files

602a4ca

Merge branch 'move-structure-rule_execution_test' of https://github.c…

8231e7c

…om/WafaaNasr/kibana into move-structure-rule_execution_test

fix conflict

36e1c63

add comma

ae3218e

merge conflict

9cd289c

run some tests in serverless

d9e187e

fix some errors of serverless

45f900b

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

a32d6c2

…ructure-rule_execution_test

add services of api integration in serverless

bdb92ca

enable query in Serverless

8172591

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

cb955fc

…ructure-rule_execution_test

remove the condition of isserverless

e8c478c

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

01ae853

…ructure-rule_execution_test

fix ignore fields

38e3dc6

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

776b7db

…ructure-rule_execution_test

remove kibana.alert.url from the expected alerts

0301147

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

bc44b91

…ructure-rule_execution_test

remove old execution config

e9681dc

WafaaNasr added 4 commits November 14, 2023 16:43

remove loading of alerts in 8.1.0

4ed5e0e

fix conflict

9162f46

remove the comment above the rule_execution bec. the query test doesn…

1526cd0

…ot unload the alerts data

fix loading legacy alert and unloading it

78fcd6d

WafaaNasr marked this pull request as ready for review November 15, 2023 13:35

WafaaNasr requested review from a team as code owners November 15, 2023 13:35

WafaaNasr requested a review from e40pud November 15, 2023 13:35

WafaaNasr and others added 3 commits November 15, 2023 14:35

Merge branch 'main' into move-structure-rule_execution_test

cff6ffc

add skipInQA to failing tests in QA

905cb56

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

11435f1

…ructure-rule_execution_test

WafaaNasr mentioned this pull request Nov 15, 2023

[Security Solution] [FTR Test] Remove @skipInQA Tags from Tests #169770

Open

WafaaNasr and others added 5 commits November 15, 2023 16:59

fix alert ancestor index name in ess

ef7229d

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

efb7210

…ructure-rule_execution_test

fix alert ancestors

c4e5aa6

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

c48c7e4

…ructure-rule_execution_test

Merge branch 'main' into move-structure-rule_execution_test

34ea580

e40pud approved these changes Nov 16, 2023

View reviewed changes

WafaaNasr added 3 commits November 16, 2023 14:04

apply comments

80f5368

Merge branch 'move-structure-rule_execution_test' of https://github.c…

5434795

…om/WafaaNasr/kibana into move-structure-rule_execution_test

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

fde47c6

…ructure-rule_execution_test

jbudz approved these changes Nov 16, 2023

View reviewed changes

WafaaNasr merged commit 2b136a2 into elastic:main Nov 16, 2023
30 checks passed

WafaaNasr deleted the move-structure-rule_execution_test branch November 16, 2023 15:14

kibanamachine added v8.12.0 backport:skip This commit does not require backporting labels Nov 16, 2023

WafaaNasr mentioned this pull request Nov 29, 2023

[Security Solution] Enable API integration tests in Serverless: Detection Engine area #164445

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][API testing] Move and restructures Rule execution logic #170765

[Security Solution][API testing] Move and restructures Rule execution logic #170765

WafaaNasr commented Nov 7, 2023 •

edited

e40pud left a comment

e40pud Nov 16, 2023

WafaaNasr Nov 16, 2023

e40pud Nov 16, 2023

WafaaNasr Nov 16, 2023

e40pud Nov 16, 2023

WafaaNasr Nov 16, 2023

e40pud Nov 16, 2023

e40pud Nov 16, 2023

kibana-ci commented Nov 16, 2023

jbudz left a comment

[Security Solution][API testing] Move and restructures Rule execution logic #170765

[Security Solution][API testing] Move and restructures Rule execution logic #170765

Conversation

WafaaNasr commented Nov 7, 2023 • edited

Summary

e40pud left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Nov 16, 2023

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

History

jbudz left a comment

Choose a reason for hiding this comment

WafaaNasr commented Nov 7, 2023 •

edited