Skip to content

Commit

Permalink
[Security Solution][API testing] Move and restructures Lists APIS (el…
Browse files Browse the repository at this point in the history
…astic#171992)

## Summary

Following the initial work in this
elastic#166755
- Addresses part of elastic#151902 for
List APIs tests
- Added a new folder under the `security_solution_api_integration`
called `lists_and_exception_lists` to hold the lists and exception lists
tests, and split the `List` APIs into two groups since the execution
time in Serverless was close to 30 mins
- Modified the
[x-pack/plugins/lists/common/schemas/response/exception_list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-4e3545fdeb8c8d9467cfa1c4aa88194e189193a92fa6f1cf5f859b1ef1beb45c),
[x-pack/plugins/lists/common/schemas/response/exception_list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-c3cc18faf07aab86e307185d41599c3596a3f8b360d3e4829591afa148283238)
,
[x-pack/plugins/lists/common/schemas/response/list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-608579ca5e65da74f41319a58d81ab12cc3d79d389b087806c7b74949fbc6cc3),
[x-pack/plugins/lists/common/schemas/response/list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-efc64eb35937a8da28fc982c527253c0923650ae4163d4bbc203d3ebc2949835)
to accept `elastic user` input because it changes in ESS and Serverless
- Deleted the `x-pack/test/lists_api_integration` folder
- Moved the utility files associated with Basic tests to the new
directory `security_solution_api_integration`. Files not actively used
in the previous folder were moved, while duplicate files remained in
their original positions.
- Updated the below files imports from the old `lists_api_integration`
folder to the new `lists_and_exception_lists`

    ```
      lists_api_integration/  
        - exceptions/operators_data_types/date_numeric_types/date.ts
        - exceptions/operators_data_types/date_numeric_types/double.ts
        - exceptions/operators_data_types/date_numeric_types/float.ts 
        - exceptions/operators_data_types/date_numeric_types/integer.ts
        - exceptions/operators_data_types/ips/ip.ts
        - exceptions/operators_data_types/ips/ip_array.ts
        - exceptions/operators_data_types/keyword/keyword.ts
        - exceptions/operators_data_types/keyword/keyword_array.ts
        - exceptions/operators_data_types/long/long.ts
        - exceptions/operators_data_types/text/text.ts
        - exceptions/operators_data_types/text/text_array.ts
        - exceptions/workflows/create_endpoint_exceptions.ts
        - exceptions/workflows/create_rule_exceptions.ts
        - exceptions/workflows/find_rule_exception_references.ts
        - exceptions/workflows/role_based_add_edit_comments.ts
        - exceptions/workflows/role_based_rule_exceptions_workflows.ts
        - exceptions/workflows/rule_exception_synchronizations.ts
        - rule_execution_logic/execution_logic/esql.ts
        - rule_execution_logic/execution_logic/machine_learning.ts
        - rule_execution_logic/execution_logic/new_terms.ts
        - rule_execution_logic/execution_logic/query.ts
        - telemetry/task_based/all_types.ts
        - telemetry/task_based/detection_rules.ts
        - telemetry/task_based/security_lists.ts
     ```
     ```
These files should be moved too soon to the new `lists_api_integration`
        detection_engine_api_integration/security_and_spaces/group10
            - import_export_rules.ts 
            - import_rules.ts
            - perform_bulk_actions   
     ```

- Updated the below files imports to the
`ftr_provider_context_with_spaces.d.ts`
` - risk_engine/risk_scoring_task/task_execution_nondefault_spaces.ts`

- The QA phase concluded with all tests passing successfully. 🟢  

- Updated the CodeOwner file for the newly moved tests

- Add a new util file to `deleteAllExceptions` under the old
`detection_engine_api` folder since the Rule management related-tests
are still need to be moved over to the new folder

- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
  • Loading branch information
WafaaNasr and kibanamachine committed Nov 28, 2023
1 parent 348ef4e commit a4ed14b
Show file tree
Hide file tree
Showing 96 changed files with 1,128 additions and 703 deletions.
6 changes: 5 additions & 1 deletion .buildkite/ftr_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,6 @@ enabled:
- x-pack/test/kubernetes_security/basic/config.ts
- x-pack/test/licensing_plugin/config.public.ts
- x-pack/test/licensing_plugin/config.ts
- x-pack/test/lists_api_integration/security_and_spaces/config.ts
- x-pack/test/monitoring_api_integration/config.ts
- x-pack/test/observability_api_integration/basic/config.ts
- x-pack/test/observability_api_integration/trial/config.ts
Expand Down Expand Up @@ -492,4 +491,9 @@ enabled:
- x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/telemetry/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/detection_engine/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/detection_engine/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/exception_lists_items/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/exception_lists_items/configs/ess.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/lists_items/configs/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/default_license/lists_items/configs/ess.config.ts


14 changes: 7 additions & 7 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -1223,7 +1223,6 @@ x-pack/plugins/cloud_integrations/cloud_full_story/server/config.ts @elastic/kib
/x-pack/test/functional/es_archives/endpoint/ @elastic/security-solution
/x-pack/test/plugin_functional/test_suites/resolver/ @elastic/security-solution
/x-pack/test/detection_engine_api_integration @elastic/security-solution
/x-pack/test/lists_api_integration @elastic/security-solution
/x-pack/test/api_integration/apis/security_solution @elastic/security-solution
#CC# /x-pack/plugins/security_solution/ @elastic/security-solution

Expand Down Expand Up @@ -1394,13 +1393,14 @@ x-pack/test/security_solution_cypress/cypress/tasks/expandable_flyout @elastic/
/x-pack/test/security_solution_cypress/cypress/e2e/entity_analytics @elastic/security-detection-engine
/x-pack/test/security_solution_cypress/cypress/e2e/exceptions @elastic/security-detection-engine
/x-pack/test/security_solution_cypress/cypress/e2e/overview @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/user_roles @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/detection_engine @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_creation @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/actions @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/alerts @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/user_roles @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/detections_response/basic_essentials_license/detection_engine @elastic/security-detection-engine
/x-pack/plugins/security_solution/server/lib/detection_engine/scripts/roles_users @elastic/security-detection-engine
/x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists @elastic/security-detection-engine

## Security Threat Intelligence - Under Security Platform
/x-pack/plugins/security_solution/public/common/components/threat_match @elastic/security-detection-engine
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,18 +53,19 @@ export const getExceptionListItemSchemaMock = (
* This is useful for end to end tests where we remove the auto generated parts for comparisons
* such as created_at, updated_at, and id.
*/
export const getExceptionListItemResponseMockWithoutAutoGeneratedValues =
(): Partial<ExceptionListItemSchema> => ({
comments: [],
created_by: ELASTIC_USER,
description: DESCRIPTION,
entries: ENTRIES,
item_id: ITEM_ID,
list_id: LIST_ID,
name: NAME,
namespace_type: 'single',
os_types: OS_TYPES,
tags: [],
type: ITEM_TYPE,
updated_by: ELASTIC_USER,
});
export const getExceptionListItemResponseMockWithoutAutoGeneratedValues = (
elasticUser: string = ELASTIC_USER
): Partial<ExceptionListItemSchema> => ({
comments: [],
created_by: elasticUser,
description: DESCRIPTION,
entries: ENTRIES,
item_id: ITEM_ID,
list_id: LIST_ID,
name: NAME,
namespace_type: 'single',
os_types: OS_TYPES,
tags: [],
type: ITEM_TYPE,
updated_by: elasticUser,
});
Original file line number Diff line number Diff line change
Expand Up @@ -82,17 +82,18 @@ export const getTrustedAppsListSchemaMock = (): ExceptionListSchema => {
* This is useful for end to end tests where we remove the auto generated parts for comparisons
* such as created_at, updated_at, and id.
*/
export const getExceptionResponseMockWithoutAutoGeneratedValues =
(): Partial<ExceptionListSchema> => ({
created_by: ELASTIC_USER,
description: DESCRIPTION,
immutable: IMMUTABLE,
list_id: LIST_ID,
name: NAME,
namespace_type: 'single',
os_types: [],
tags: [],
type: ENDPOINT_TYPE,
updated_by: ELASTIC_USER,
version: VERSION,
});
export const getExceptionResponseMockWithoutAutoGeneratedValues = (
elasticUser: string = ELASTIC_USER
): Partial<ExceptionListSchema> => ({
created_by: elasticUser,
description: DESCRIPTION,
immutable: IMMUTABLE,
list_id: LIST_ID,
name: NAME,
namespace_type: 'single',
os_types: [],
tags: [],
type: ENDPOINT_TYPE,
updated_by: elasticUser,
version: VERSION,
});
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,12 @@ export const getListItemResponseMock = (): ListItemSchema => ({
* This is useful for end to end tests where we remove the auto generated parts for comparisons
* such as created_at, updated_at, and id.
*/
export const getListItemResponseMockWithoutAutoGeneratedValues = (): Partial<ListItemSchema> => ({
created_by: ELASTIC_USER,
export const getListItemResponseMockWithoutAutoGeneratedValues = (
elasticUser: string = ELASTIC_USER
): Partial<ListItemSchema> => ({
created_by: elasticUser,
list_id: LIST_ID,
type: TYPE,
updated_by: ELASTIC_USER,
updated_by: elasticUser,
value: VALUE,
});
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,14 @@ export const getListResponseMock = (): ListSchema => ({
* This is useful for end to end tests where we remove the auto generated parts for comparisons
* such as created_at, updated_at, and id.
*/
export const getListResponseMockWithoutAutoGeneratedValues = (): Partial<ListSchema> => ({
created_by: ELASTIC_USER,
export const getListResponseMockWithoutAutoGeneratedValues = (
elasticUser: string = ELASTIC_USER
): Partial<ListSchema> => ({
created_by: elasticUser,
description: DESCRIPTION,
immutable: IMMUTABLE,
name: NAME,
type: TYPE,
updated_by: ELASTIC_USER,
updated_by: elasticUser,
version: VERSION,
});
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ import {
deleteAllRules,
deleteAllAlerts,
getSimpleRule,
deleteAllExceptions,
} from '../../utils';
import { deleteAllExceptions } from '../../../lists_api_integration/utils';
import { createUserAndRole, deleteUserAndRole } from '../../../common/services/security_solution';

// This test was meant to be more full flow, ensuring that
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ import {
createRule,
getRule,
getRuleSOById,
deleteAllExceptions,
} from '../../utils';
import { deleteAllExceptions } from '../../../lists_api_integration/utils';
import { createUserAndRole, deleteUserAndRole } from '../../../common/services/security_solution';

const getImportRuleBuffer = (connectorId: string) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import { getCreateExceptionListDetectionSchemaMock } from '@kbn/lists-plugin/com
import { EXCEPTION_LIST_ITEM_URL, EXCEPTION_LIST_URL } from '@kbn/securitysolution-list-constants';
import { getCreateExceptionListItemMinimalSchemaMock } from '@kbn/lists-plugin/common/schemas/request/create_exception_list_item_schema.mock';
import { WebhookAuthType } from '@kbn/stack-connectors-plugin/common/webhook/constants';
import { deleteAllExceptions } from '../../../lists_api_integration/utils';
import {
binaryToString,
createLegacyRuleAction,
Expand All @@ -44,6 +43,7 @@ import {
createRuleThroughAlertingEndpoint,
getRuleSavedObjectWithLegacyInvestigationFields,
getRuleSavedObjectWithLegacyInvestigationFieldsEmptyArray,
deleteAllExceptions,
} from '../../utils';
import { FtrProviderContext } from '../../common/ftr_provider_context';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ import {
getWebHookAction,
removeServerGeneratedProperties,
ruleToNdjson,
deleteAllExceptions,
} from '../../utils';
import { deleteAllExceptions } from '../../../lists_api_integration/utils';
import { createUserAndRole, deleteUserAndRole } from '../../../common/services/security_solution';

const getImportRuleBuffer = (connectorId: string) => {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

// Should be deleted once all all the remaining tests in this folder get moved to the new /security_solution_api_integration folder

import type SuperTest from 'supertest';

import type { ExceptionList, NamespaceType } from '@kbn/securitysolution-io-ts-list-types';
import { EXCEPTION_LIST_URL } from '@kbn/securitysolution-list-constants';

import { ToolingLog } from '@kbn/tooling-log';
import { countDownTest } from './count_down_test';

/**
* Remove all exceptions from both the "single" and "agnostic" spaces.
* This will retry 50 times before giving up and hopefully still not interfere with other tests
* @param supertest The supertest handle
*/
export const deleteAllExceptions = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog
): Promise<void> => {
await deleteAllExceptionsByType(supertest, log, 'single');
await deleteAllExceptionsByType(supertest, log, 'agnostic');
};

/**
* Remove all exceptions by a given type such as "agnostic" or "single".
* This will retry 50 times before giving up and hopefully still not interfere with other tests
* @param supertest The supertest handle
*/
export const deleteAllExceptionsByType = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
type: NamespaceType
): Promise<void> => {
await countDownTest(
async () => {
const { body } = await supertest
.get(`${EXCEPTION_LIST_URL}/_find?per_page=9999&namespace_type=${type}`)
.set('kbn-xsrf', 'true')
.send();
const ids: string[] = body.data.map((exception: ExceptionList) => exception.id);
for await (const id of ids) {
await supertest
.delete(`${EXCEPTION_LIST_URL}?id=${id}&namespace_type=${type}`)
.set('kbn-xsrf', 'true')
.send();
}
const { body: finalCheck } = await supertest
.get(`${EXCEPTION_LIST_URL}/_find?namespace_type=${type}`)
.set('kbn-xsrf', 'true')
.send();
return {
passed: finalCheck.data.length === 0,
};
},
`deleteAllExceptions by type: "${type}"`,
log,
50,
1000
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -86,3 +86,4 @@ export * from './prebuilt_rules/delete_all_prebuilt_rule_assets';
export * from './prebuilt_rules/install_mock_prebuilt_rules';
export * from './prebuilt_rules/install_prebuilt_rules_and_timelines';
export * from './get_legacy_action_so';
export * from './delete_all_exceptions';
66 changes: 0 additions & 66 deletions x-pack/test/lists_api_integration/common/config.ts

This file was deleted.

12 changes: 0 additions & 12 deletions x-pack/test/lists_api_integration/common/ftr_provider_context.d.ts

This file was deleted.

8 changes: 0 additions & 8 deletions x-pack/test/lists_api_integration/common/services.ts

This file was deleted.

0 comments on commit a4ed14b

Please sign in to comment.