[Security Solution][API testing] Move and restructures Alerts' tests #170350

WafaaNasr · 2023-11-01T19:26:47Z

Summary

Following the initial work in this [Security Solution][Detection Engine] Running API tests in Serverless & ESS using Mocha Tagging #166755
Addresses part of [Security Solution] Restructure API integration tests into meaningful groups that run in parallel on CI #151902 for Alerts
Introduced a new folder called alerts under security_solution_api_integration
Moved the utility files associated with alerts to the new directory security_solution_api_integration. Files that were not actively used in the previous folder were moved, while any duplicate files remained in their original positions.
Updated the CodeOwner file for the newly moved tests
Old / new groups details and execution time document

Action	File	New Path if moved
Delete	group1/find_rule_exception_references.ts	Already moved in previous PR
Delete	security_and_spaces/group6	-
Move	detection_engine_api_integration/security_and_spaces/group6	detections_response/default_license/alerts/alerts_compatibility.ts
Move	detection_engine_api_integration/security_and_spaces/group1	detections_response/default_license/alerts/aliases.ts
Move	detection_engine_api_integration/security_and_spaces/group1	detections_response/default_license/alerts/create_index.ts
Move	detection_engine_api_integration/security_and_spaces/group10/create_signals_migrations	detections_response/default_license/alerts/migrations/create_alerts_migrations.ts
Move	detection_engine_api_integration/security_and_spaces/group10/delete_signals_migrations	detections_response/default_license/alerts/migrations/delete_alerts_migrations.ts
Move	detection_engine_api_integration/security_and_spaces/group10/finalize_signals_migrations	detections_response/default_license/alerts/migrations/finalize_alerts_migrations.ts
Move	detection_engine_api_integration/security_and_spaces/group10/get_signals_migration_status	detections_response/default_license/alerts/migrations/get_alerts_migration_status.ts
Move	detection_engine_api_integration/security_and_spaces/group10/open_close_signals	detections_response/default_license/alerts/open_close_alerts.ts
Move	detection_engine_api_integration/security_and_spaces/group10/alert_tags	detections_response/default_license/alerts/alert_tags.ts

Note: these tests are skipped on the main branch

https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/open_close_signals.ts#L215

https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/open_close_signals.ts#L252

https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/finalize_signals_migrations.ts#L192

https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group1/create_index.ts#L42

…fix'

…asr/kibana into move-structure-alerts-api

…ructure-alerts-api

…asr/kibana into move-structure-alerts-api

yctercero · 2023-11-03T22:05:12Z

.github/CODEOWNERS

+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions  @elastic/security-detection-engine
+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation @elastic/security-detection-engine
+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions @elastic/security-detection-engine
+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation @elastic/security-detection-engine


I think this is a duplicate of line 1315?

Correct! replaced it with alerts, Thank you!

…ructure-alerts-api

rylnd

Changes here look correct, and CI seems to confirm things are working. I had a few nits about the names of these new test suites, but overall LGTM.

rylnd · 2023-11-06T20:52:01Z

.github/CODEOWNERS

+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation @elastic/security-detection-engine
+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions @elastic/security-detection-engine
+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts @elastic/security-detection-engine
+x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts @elastic/security-detection-engine


Duplicate line?

Please find it fixed here

rylnd · 2023-11-06T20:56:33Z

...api_integration/test_suites/detections_response/default_license/alerts/configs/ess.config.ts

-    testFiles: [require.resolve('.')],
+    testFiles: [require.resolve('..')],
+    junit: {
+      reportName: 'Detection Engine ESS/Alerts API Integration Tests',


Nit: the slash here makes it look like "ESS/Alerts" is a concept. These seem to be pretty freeform, so I'll throw out a suggestion:

Suggested change

reportName: 'Detection Engine ESS/Alerts API Integration Tests',

reportName: 'Detection Engine API Integration Tests - ESS - Alerts ',

Makes sense to me! I changed it here for the Alerts now and then I will iterate all of them too
Thanks Ryland!!
Please find it fixed here

rylnd · 2023-11-06T20:56:48Z

...egration/test_suites/detections_response/default_license/alerts/configs/serverless.config.ts

+export default createTestConfig({
+  testFiles: [require.resolve('..')],
+  junit: {
+    reportName: 'Detection Engine Serverless/Alerts API Integration Tests',


ditto here on the name

Please find it fixed here

rylnd · 2023-11-06T20:57:26Z

...est_suites/detections_response/default_license/alerts/migrations/create_alerts_migrations.ts

@@ -35,7 +39,6 @@ function sleep(ms: number) {
  return new Promise((resolve) => setTimeout(resolve, ms));
 }

-// eslint-disable-next-line import/no-default-export


rylnd · 2023-11-06T21:01:33Z

x-pack/test/tsconfig.json

@@ -106,7 +106,6 @@
    "@kbn/journeys",
    "@kbn/stdio-dev-helpers",
    "@kbn/alerting-api-integration-helpers",
-    "@kbn/securitysolution-ecs",


Clarification: is this being removed since only the child security_solution_api_integration now references it?

Usually these changes are done by the CI, so I am not 100% sure but I kinda have the same explanation since it used only in the new security_solution_api_integration folder now

…ructure-alerts-api

kibana-ci · 2023-11-07T12:36:52Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: d92242e

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #67 / Serverless Common UI - Platform Security API keys "before all" hook for "should create and delete API keys correctly"

Metrics [docs]

✅ unchanged

History

💔 Build #173716 failed 8da91ae
💛 Build #173387 was flaky da16895
💛 Build #173246 was flaky a6167c5
💚 Build #172916 succeeded 6e8d741
💔 Build #172871 failed 75050f2
💔 Build #172827 failed 1f11162

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @WafaaNasr

…lastic#170350) ## Summary - Following the initial work in this elastic#166755 - Addresses part of elastic#151902 for Alerts - Introduced a new folder called `alerts` under `security_solution_api_integration` - Moved the utility files associated with alerts to the new directory `security_solution_api_integration`. Files that were not actively used in the previous folder were moved, while any duplicate files remained in their original positions. - Updated the CodeOwner file for the newly moved tests - Old / new groups details and execution time [document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit) | Action | File | New Path if moved | |--------|------|----------| | Delete| group1/find_rule_exception_references.ts| Already moved in previous PR | | Delete|security_and_spaces/group6 | - | | Move|detection_engine_api_integration/security_and_spaces/group6 |detections_response/default_license/alerts/alerts_compatibility.ts| | Move|detection_engine_api_integration/security_and_spaces/group1| detections_response/default_license/alerts/aliases.ts | | Move |detection_engine_api_integration/security_and_spaces/group1| detections_response/default_license/alerts/create_index.ts| | Move|detection_engine_api_integration/security_and_spaces/group10/create_signals_migrations| detections_response/default_license/alerts/migrations/create_alerts_migrations.ts | | Move|detection_engine_api_integration/security_and_spaces/group10/delete_signals_migrations| detections_response/default_license/alerts/migrations/delete_alerts_migrations.ts| | Move|detection_engine_api_integration/security_and_spaces/group10/finalize_signals_migrations| detections_response/default_license/alerts/migrations/finalize_alerts_migrations.ts | | Move|detection_engine_api_integration/security_and_spaces/group10/get_signals_migration_status| detections_response/default_license/alerts/migrations/get_alerts_migration_status.ts| | Move|detection_engine_api_integration/security_and_spaces/group10/open_close_signals| detections_response/default_license/alerts/open_close_alerts.ts| | Move|detection_engine_api_integration/security_and_spaces/group10/alert_tags| detections_response/default_license/alerts/alert_tags.ts| Note: these tests are skipped on the main branch https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/open_close_signals.ts#L215 https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/open_close_signals.ts#L252 https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/finalize_signals_migrations.ts#L192 https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group1/create_index.ts#L42 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

move the migrations of alerts

571fc6d

WafaaNasr self-assigned this Nov 1, 2023

WafaaNasr and others added 8 commits November 1, 2023 20:27

fix conflict

1a888b0

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

b984b22

…fix'

remove old exports

fa594e5

Merge branch 'move-structure-alerts-api' of https://github.com/WafaaN…

72d5767

…asr/kibana into move-structure-alerts-api

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

a4edfdf

…ructure-alerts-api

move open_close_alert and alert_tags

6bfc074

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

1f11162

…ructure-alerts-api

undo remove export of query_signal

75050f2

WafaaNasr added release_note:skip Skip the PR/issue when compiling release notes FTR labels Nov 2, 2023

WafaaNasr marked this pull request as ready for review November 2, 2023 16:47

WafaaNasr requested review from a team as code owners November 2, 2023 16:47

Merge branch 'main' into move-structure-alerts-api

6e8d741

jbudz approved these changes Nov 2, 2023

View reviewed changes

WafaaNasr added 2 commits November 3, 2023 18:49

fix conflict

c600db7

Merge branch 'move-structure-alerts-api' of https://github.com/WafaaN…

a6167c5

…asr/kibana into move-structure-alerts-api

yctercero reviewed Nov 3, 2023

View reviewed changes

WafaaNasr added 2 commits November 6, 2023 10:19

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

6faae04

…ructure-alerts-api

fix codeowner alerts ownership

da16895

rylnd approved these changes Nov 6, 2023

View reviewed changes

WafaaNasr and others added 3 commits November 7, 2023 10:33

apply comments

d2b5fb5

Merge branch 'main' of https://github.com/elastic/kibana into move-st…

8da91ae

…ructure-alerts-api

Merge branch 'main' into move-structure-alerts-api

d92242e

WafaaNasr merged commit 2b5f4f7 into elastic:main Nov 7, 2023
28 checks passed

WafaaNasr deleted the move-structure-alerts-api branch November 7, 2023 12:42

kibanamachine added v8.12.0 backport:skip This commit does not require backporting labels Nov 7, 2023

WafaaNasr mentioned this pull request Nov 29, 2023

[Security Solution] Enable API integration tests in Serverless: Detection Engine area #164445

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][API testing] Move and restructures Alerts' tests #170350

[Security Solution][API testing] Move and restructures Alerts' tests #170350

WafaaNasr commented Nov 1, 2023 •

edited

yctercero Nov 3, 2023

WafaaNasr Nov 6, 2023

rylnd left a comment

rylnd Nov 6, 2023

WafaaNasr Nov 7, 2023

rylnd Nov 6, 2023

WafaaNasr Nov 7, 2023

rylnd Nov 6, 2023

WafaaNasr Nov 7, 2023

rylnd Nov 6, 2023

rylnd Nov 6, 2023

WafaaNasr Nov 7, 2023

kibana-ci commented Nov 7, 2023

	reportName: 'Detection Engine ESS/Alerts API Integration Tests',
	reportName: 'Detection Engine API Integration Tests - ESS - Alerts ',

[Security Solution][API testing] Move and restructures Alerts' tests #170350

[Security Solution][API testing] Move and restructures Alerts' tests #170350

Conversation

WafaaNasr commented Nov 1, 2023 • edited

Summary

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rylnd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Nov 7, 2023

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

WafaaNasr commented Nov 1, 2023 •

edited