[Security Solution][API testing] Move and restructures Risk Engine te…

…sts (elastic#170530) ## Summary Following the initial work in this elastic#166755 - Addresses part of elastic#151902 for Entity Analytics/Risk Engine - Introduced new folders called `entity_analytics/risk_engine` under `security_solution_api_integration` - Moved the utility files associated with risk_engine to the new directory `security_solution_api_integration`. Files that were not actively used in the previous folder were moved, while any duplicate files remained in their original positions. - Updated the CodeOwner file for the newly moved tests - Old / new groups details and execution time [document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit) | Action | File | New Path if moved | |--------|------|----------| | Moved| group10/risk_engine/init_and_status_apis|/entity_analytics/default_license/risk_engine/init_and_status_apis.ts | | Moved| group10/risk_engine/risk_score_calculation|/entity_analytics/default_license/risk_engine/risk_score_calculation.ts | | Moved| group10/risk_engine/risk_score_preview|/entity_analytics/default_license/risk_engine/risk_score_preview.ts | | Moved| group10/risk_engine/risk_scoring_task_execution|/entity_analytics/default_license/risk_engine/risk_scoring_task_execution.ts | | Moved| group10/risk_engine/telemetry_usage|/entity_analytics/default_license/risk_engine/telemetry_usage.ts | | Moved| group10/risk_engine/utils|entity_analytics/utils/risk_engine.ts | | Moved| utils/get_stats|entity_analytics/utils/get_risk_engine_stats.ts | Tests skipped on main: https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/init_and_status_apis.ts#L363 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Nikita Khristinin <nkhristinin@gmail.com>
tkajtoch · Nov 9, 2023 · 15e8cef · 15e8cef
1 parent 23617bf
commit 15e8cef
Show file tree

Hide file tree

Showing 33 changed files with 1,335 additions and 172 deletions.
diff --git a/.buildkite/ftr_configs.yml b/.buildkite/ftr_configs.yml
@@ -462,6 +462,9 @@ enabled:
   - x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions/configs/ess.config.ts
   - x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts/configs/serverless.config.ts
   - x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts/configs/ess.config.ts
+  - x-pack/test/security_solution_api_integration/test_suites/entity_analytics/default_license/risk_engine/configs/serverless.config.ts
+  - x-pack/test/security_solution_api_integration/test_suites/entity_analytics/default_license/risk_engine/configs/ess.config.ts
+
 
 
 

diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -1438,7 +1438,7 @@ x-pack/plugins/security_solution/public/overview/pages/entity_analytics.tsx @ela
 x-pack/plugins/security_solution/public/overview/components/entity_analytics
 x-pack/plugins/security_solution/server/lib/entity_analytics @elastic/security-entity-analytics
 x-pack/plugins/security_solution/server/lib/risk_score @elastic/security-entity-analytics
-x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine @elastic/security-entity-analytics
+x-pack/test/security_solution_api_integration/test_suites/entity_analytics @elastic/security-entity-analytics
 
 # Security Defend Workflows - OSQuery Ownership
 /x-pack/plugins/security_solution/common/api/detection_engine/model/rule_response_actions @elastic/security-defend-workflows

diff --git a/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/index.ts b/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/index.ts
@@ -30,10 +30,5 @@ export default ({ loadTestFile }: FtrProviderContext): void => {
     loadTestFile(require.resolve('./runtime'));
     loadTestFile(require.resolve('./throttle'));
     loadTestFile(require.resolve('./ignore_fields'));
-    loadTestFile(require.resolve('./risk_engine/init_and_status_apis'));
-    loadTestFile(require.resolve('./risk_engine/risk_score_preview'));
-    loadTestFile(require.resolve('./risk_engine/risk_score_calculation'));
-    loadTestFile(require.resolve('./risk_engine/risk_scoring_task_execution'));
-    loadTestFile(require.resolve('./risk_engine/telemetry_usage'));
   });
 };
diff --git a/x-pack/test/security_solution_api_integration/README.md b/x-pack/test/security_solution_api_integration/README.md
@@ -28,65 +28,71 @@ ex:
 
  ```
 
-## Adding new security area's tests
+# Adding new security area's tests
 
 1. Within the `test_suites` directory, create a new area folder.
 2. Introduce `ess.config` and `serverless.config` files to reference the new test files and incorporate any additional custom properties defined in the `CreateTestConfigOptions` interface.
 3. In these new configuration files, include references to the base configurations located under the config directory to inherit CI configurations, environment variables, and other settings.
 4. Append a new entry in the `ftr_configs.yml` file to enable the execution of the newly added tests within the CI pipeline.
 
 
-## Testing locally 
+# Testing locally 
 
 In the `package.json` file, you'll find commands to configure the server for each environment and to run tests against that specific environment. These commands adhere to the Mocha tagging system, allowing for the inclusion and exclusion of tags, mirroring the setup of the CI pipeline.
 
 ## Running Commands with Different Parameters
 
 In this project, you can run various commands to execute tests and workflows, each of which can be customized by specifying different parameters. Below, how to define the commands based on the parameters and their order.
 
-### Command Structure
-
-The command structure follows this pattern:
-
-- `<command-name>`: The name of the specific command or test case.
-- `<folder>`: The test folder or workflow you want to run.
-- `<type>`: The type of operation, either "server" or "runner."
-- `<environment>`: The testing environment, such as "serverlessEnv," "essEnv," or "qaEnv."
-- `<licenseFolder>`: The license folder the test is defined under such as "default_license", by default the value is "default_license"
-- `<area>`: The area the test is defined under, such as "detection_engine", by default the value is "detection_engine"
-
-### Serverless and Ess Configuration
-
-- When using "serverless" or "ess" in the script, it specifies the correct configuration file for the tests.
-- "Serverless" and "ess" help determine the configuration specific to the chosen test.
-
-### serverlessEnv, essEnv, qaEnv Grep Command
-
-- When using "serverlessEnv,.." in the script, it appends the correct grep command for filtering tests in the serverless testing environment.
-- "serverlessEnv,..." is used to customize the test execution based on the serverless environment.
-
-
-### Command Examples
-
-Here are some command examples using the provided parameters:
-
-1. **Run the server for "exception_workflows" in the "serverlessEnv" environment:**
-   ```shell
-   npm run initialize-server exceptions/workflows serverless
-   ```
-2. **To run tests for the "exception_workflows" using the serverless runner in the "serverlessEnv" environment, you can use the following command:**
-    ```shell
-    npm run run-tests exceptions/workflows serverless serverlessEnv
-    ```
-3. **Run tests for "exception_workflows" using the serverless runner in the "qaEnv" environment:**
-    ```shell
-    npm run run-tests exceptions/workflows serverless qaEnv
-    ```
-4. **Run the server for "exception_workflows" in the "essEnv" environment:**
-   ```shell
-    npm run initialize-server exceptions/workflows ess
-   ```
-5. **Run tests for "exception_workflows" using the ess runner in the "essEnv" environment:**   
-   ```shell
-   npm run run-tests exceptions/workflows ess essEnv
-   ```
+1.  Server Initialization and running tests for ex: (Detections Response - Default License):
+
+    The command structure follows this pattern
+    - `<type>` can be either "server" or "runner," allowing you to either set up the server or execute the tests against the designated server.
+    - `<area>`: The area the test is defined under, such as "detection_engine, entity_analytics,.."
+    - `<licenseFolder>`: The license folder the test is defined under such as "default_license, basic_license,..."
+
+      #### `initialize-server:dr:default`
+
+      - Command: `node ./scripts/index.js server detections_response default_license`
+      - Description: Initiates the server for the Detections Response area with the default license.
+      #### `run-tests:dr:default`
+
+      - Command: `node ./scripts/index.js runner detections_response default_license`
+      - Description: Runs the tests for the Detections Response area with the default license.
+
+
+
+ 2. Executes particular sets of test suites linked to the designated environment and license:
+
+     The command structure follows this pattern:
+
+      - `<folder>`: The test folder or workflow you want to run.
+      - `<projectType>`: The type of project to pick the relevant configurations, either "serverless" or "ess."
+        - "serverless" and "ess" help determine the configuration specific to the chosen test.
+      - `<environment>`: The testing environment, such as "serverlessEnv," "essEnv," or "qaEnv."
+        - When using "serverlessEnv,.." in the script, it appends the correct grep command for filtering tests in the serverless  testing environment.
+        - "serverlessEnv,..." is used to customize the test execution based on the serverless environment.
+
+      
+      Here are some command examples for "exceptions" which defined under the "detection_engine" area using the default license:
+
+      1. **Run the server for "exception_workflows" in the "serverlessEnv" environment:**
+         ```shell
+         npm run initialize-server:dr:default exceptions/workflows serverless
+         ```
+      2. **To run tests for the "exception_workflows" using the serverless runner in the "serverlessEnv" environment, you can use the following command:**
+         ```shell
+         npm run run-tests:dr:default exceptions/workflows serverless serverlessEnv
+         ```
+      3. **Run tests for "exception_workflows" using the serverless runner in the "qaEnv" environment:**
+         ```shell
+         npm run run-tests:dr:default exceptions/workflows serverless qaEnv
+         ```
+      4. **Run the server for "exception_workflows" in the "essEnv" environment:**
+         ```shell
+         npm run initialize-server:dr:default exceptions/workflows ess   
+         ```
+      5. **Run tests for "exception_workflows" using the ess runner in the "essEnv" environment:**   
+         ```shell
+         npm run run-tests:dr:default exceptions/workflows ess essEnv
+      ```
diff --git a/x-pack/test/security_solution_api_integration/ftr_provider_context.d.ts b/x-pack/test/security_solution_api_integration/ftr_provider_context.d.ts
@@ -4,7 +4,8 @@
  * 2.0; you may not use this file except in compliance with the Elastic License
  * 2.0.
  */
+import { GenericFtrProviderContext } from '@kbn/test';
 
-import type { FtrProviderContext } from '../../test_serverless/api_integration/ftr_provider_context';
+import { services } from '../../test_serverless/api_integration/services';
 
-export type { FtrProviderContext };
+export type FtrProviderContext = GenericFtrProviderContext<typeof services, {}>;
diff --git a/x-pack/test/security_solution_api_integration/package.json b/x-pack/test/security_solution_api_integration/package.json
@@ -7,6 +7,8 @@
   "scripts": {
     "initialize-server:dr:default": "node ./scripts/index.js server detections_response default_license",
     "run-tests:dr:default": "node ./scripts/index.js runner detections_response default_license",
+    "initialize-server:ea:default": "node ./scripts/index.js server entity_analytics default_license",
+    "run-tests:ea:default": "node ./scripts/index.js runner entity_analytics default_license",
     "exception_workflows:server:serverless": "npm run initialize-server:dr:default exceptions/workflows serverless",
     "exception_workflows:runner:serverless": "npm run run-tests:dr:default exceptions/workflows serverless serverlessEnv",
     "exception_workflows:qa:serverless": "npm run run-tests:dr:default exceptions/workflows serverless qaEnv",
@@ -41,6 +43,11 @@
     "alerts:runner:serverless": "npm run run-tests:dr:default alerts serverless serverlessEnv",
     "alerts:qa:serverless": "npm run run-tests:dr:default alerts serverless qaEnv",
     "alerts:server:ess": "npm run initialize-server:dr:default alerts ess",
-    "alerts:runner:ess": "npm run run-tests:dr:default alerts ess essEnv"
+    "alerts:runner:ess": "npm run run-tests:dr:default alerts ess essEnv",
+    "entity_analytics:server:serverless": "npm run initialize-server:ea:default risk_engine serverless",
+    "entity_analytics:runner:serverless": "npm run run-tests:ea:default risk_engine serverless serverlessEnv",
+    "entity_analytics:qa:serverless": "npm run run-tests:ea:default risk_engine serverless qaEnv",
+    "entity_analytics:server:ess": "npm run initialize-server:ea:default risk_engine ess",
+    "entity_analytics:runner:ess": "npm run run-tests:ea:default risk_engine ess essEnv"
   }
 }
diff --git a/...olution_api_integration/test_suites/detections_response/utils/alerts/get_alerts_by_ids.ts b/...olution_api_integration/test_suites/detections_response/utils/alerts/get_alerts_by_ids.ts
@@ -14,6 +14,7 @@ import type { RiskEnrichmentFields } from '@kbn/security-solution-plugin/server/
 import { DETECTION_ENGINE_QUERY_SIGNALS_URL as DETECTION_ENGINE_QUERY_ALERTS_URL } from '@kbn/security-solution-plugin/common/constants';
 import { countDownTest } from '../count_down_test';
 import { getQueryAlertsId } from './get_query_alerts_ids';
+import { routeWithNamespace } from '../route_with_namespace';
 
 /**
  * Given an array of rule ids this will return only alerts based on that rule id both
@@ -25,12 +26,14 @@ export const getAlertsByIds = async (
   supertest: SuperTest.SuperTest<SuperTest.Test>,
   log: ToolingLog,
   ids: string[],
-  size?: number
+  size?: number,
+  namespace?: string
 ): Promise<SearchResponse<DetectionAlert & RiskEnrichmentFields>> => {
   const alertsOpen = await countDownTest<SearchResponse<DetectionAlert & RiskEnrichmentFields>>(
     async () => {
+      const route = routeWithNamespace(DETECTION_ENGINE_QUERY_ALERTS_URL, namespace);
       const response = await supertest
-        .post(DETECTION_ENGINE_QUERY_ALERTS_URL)
+        .post(route)
         .set('kbn-xsrf', 'true')
         .send(getQueryAlertsId(ids, size));
       if (response.status !== 200) {

diff --git a/...integration/test_suites/detections_response/utils/alerts/wait_for_alerts_to_be_present.ts b/...integration/test_suites/detections_response/utils/alerts/wait_for_alerts_to_be_present.ts
@@ -21,11 +21,12 @@ export const waitForAlertsToBePresent = async (
   supertest: SuperTest.SuperTest<SuperTest.Test>,
   log: ToolingLog,
   numberOfAlerts = 1,
-  alertIds: string[]
+  alertIds: string[],
+  namespace?: string
 ): Promise<void> => {
   await waitFor(
     async () => {
-      const alertsOpen = await getAlertsByIds(supertest, log, alertIds, numberOfAlerts);
+      const alertsOpen = await getAlertsByIds(supertest, log, alertIds, numberOfAlerts, namespace);
       return alertsOpen.hits.hits.length >= numberOfAlerts;
     },
     'waitForAlertsToBePresent',