Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Add missing Exceptions API OpenAPI specifications #185951

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

maximpn
Copy link
Contributor

@maximpn maximpn commented Jun 11, 2024

Resolves: #183837

Summary

This PR adds missing OpenAPI specifications for Exceptions API which are the following

  • POST /api/exception_lists/_export
  • POST /api/exception_lists/_import
  • POST /api/exception_lists
  • GET /api/exception_lists
  • PUT /api/exception_lists
  • DELETE /api/exception_lists
  • GET /api/exception_lists/_find
  • POST /api/exception_lists/_duplicate
  • POST /api/exception_lists/items
  • GET /api/exception_lists/items
  • PUT /api/exception_lists/items
  • DELETE /api/exception_lists/items
  • GET /api/exception_lists/items/_find
  • GET /api/exception_lists/summary
  • POST /api/exceptions/shared
  • POST /api/detection_engine/rules/{id}/exceptions

@maximpn maximpn added release_note:skip Skip the PR/issue when compiling release notes docs Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Team:Detection Engine Security Solution Detection Engine Area v8.15.0 labels Jun 11, 2024
@maximpn maximpn self-assigned this Jun 11, 2024
@maximpn maximpn force-pushed the add-missing-exceptions-api-oas branch 2 times, most recently from 22d2fc8 to 61ad063 Compare June 12, 2024 14:03
@maximpn maximpn force-pushed the add-missing-exceptions-api-oas branch 11 times, most recently from d4d7cd8 to 09f4a03 Compare June 21, 2024 20:40
@maximpn maximpn marked this pull request as ready for review June 22, 2024 05:28
@maximpn maximpn requested review from a team as code owners June 22, 2024 05:28
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@maximpn maximpn requested a review from rylnd June 22, 2024 05:28
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-engine (Team:Detection Engine)

@maximpn maximpn requested review from banderror and marshallmain and removed request for rylnd June 22, 2024 05:29
@maximpn maximpn requested a review from a team as a code owner June 22, 2024 06:04
@maximpn maximpn force-pushed the add-missing-exceptions-api-oas branch from aeb4a00 to d51ce47 Compare June 22, 2024 08:03
@maximpn maximpn requested a review from a team as a code owner June 22, 2024 12:30
@maximpn maximpn force-pushed the add-missing-exceptions-api-oas branch 2 times, most recently from 4de0de3 to 632c69a Compare June 22, 2024 19:20
@maximpn
Copy link
Contributor Author

maximpn commented Jun 22, 2024

Files by Code Owner

elastic/kibana-operations

  • .buildkite/scripts/steps/code_generation/security_solution_codegen.sh

elastic/security-defend-workflows

  • x-pack/test/security_solution_api_integration/test_suites/security_solution_endpoint_api_int/apis/endpoint_artifacts/blocklists.ts

elastic/security-detection-engine

  • packages/kbn-securitysolution-io-ts-list-types/src/request/index.ts
  • packages/kbn-securitysolution-io-ts-list-types/src/request/update_exception_list_item_schema/index.ts
  • packages/kbn-securitysolution-io-ts-list-types/src/request/update_exception_list_item_validation/index.test.ts
  • packages/kbn-securitysolution-io-ts-list-types/src/request/update_exception_list_item_validation/index.ts
  • packages/kbn-securitysolution-io-ts-list-types/tsconfig.json
  • packages/kbn-securitysolution-lists-common/scripts/openapi_generate.js
  • x-pack/plugins/lists/common/api/exceptions/create_exception_list_item/create_exception_list_item_route.ts
  • x-pack/plugins/lists/common/api/exceptions/delete_exception_list/delete_exception_list_route.ts
  • x-pack/plugins/lists/common/api/exceptions/delete_exception_list_item/delete_exception_list_item_route.ts
  • x-pack/plugins/lists/common/api/exceptions/duplicate_exception_list/duplicate_exception_list_route.ts
  • x-pack/plugins/lists/common/api/exceptions/export_exception_list/export_exception_list_route.ts
  • x-pack/plugins/lists/common/api/exceptions/export_list_item/export_list_item_route.ts
  • x-pack/plugins/lists/common/api/exceptions/find_exception_list/find_exception_list_route.ts
  • x-pack/plugins/lists/common/api/exceptions/find_exception_list_item/find_exception_list_item_route.ts
  • x-pack/plugins/lists/common/api/exceptions/import_exceptions/import_exceptions_route.ts
  • x-pack/plugins/lists/common/api/exceptions/read_exception_list/read_exception_list_route.ts
  • x-pack/plugins/lists/common/api/exceptions/read_exception_list_item/read_exception_list_item_route.ts
  • x-pack/plugins/lists/common/api/exceptions/summary_exception_list/summary_exception_list_route.ts
  • x-pack/plugins/lists/common/api/exceptions/update_exception_list/update_exception_list_route.ts
  • x-pack/plugins/lists/common/api/exceptions/update_exception_list_item/update_exception_list_item_route.ts
  • x-pack/plugins/lists/common/api/index.ts
  • x-pack/plugins/lists/server/routes/create_exception_list_item_route.ts
  • x-pack/plugins/lists/server/routes/create_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/delete_exception_list_item_route.ts
  • x-pack/plugins/lists/server/routes/delete_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/duplicate_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/export_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/find_exception_list_item_route.ts
  • x-pack/plugins/lists/server/routes/find_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/import_exceptions_route.ts
  • x-pack/plugins/lists/server/routes/list/create_list_route.ts
  • x-pack/plugins/lists/server/routes/list/delete_list_route.ts
  • x-pack/plugins/lists/server/routes/list/patch_list_route.ts
  • x-pack/plugins/lists/server/routes/list/read_list_route.ts
  • x-pack/plugins/lists/server/routes/list/update_list_route.ts
  • x-pack/plugins/lists/server/routes/list_index/export_list_item_route.ts
  • x-pack/plugins/lists/server/routes/list_index/find_list_route.ts
  • x-pack/plugins/lists/server/routes/list_item/create_list_item_route.ts
  • x-pack/plugins/lists/server/routes/list_item/delete_list_item_route.ts
  • x-pack/plugins/lists/server/routes/list_item/find_list_item_route.ts
  • x-pack/plugins/lists/server/routes/list_item/patch_list_item_route.ts
  • x-pack/plugins/lists/server/routes/list_item/read_list_item_route.ts
  • x-pack/plugins/lists/server/routes/list_item/update_list_item_route.ts
  • x-pack/plugins/lists/server/routes/read_exception_list_item_route.ts
  • x-pack/plugins/lists/server/routes/read_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/summary_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/update_exception_list_item_route.ts
  • x-pack/plugins/lists/server/routes/update_exception_list_route.ts
  • x-pack/plugins/lists/server/routes/utils/validate_comments_to_update.test.ts
  • x-pack/plugins/lists/server/routes/utils/validate_comments_to_update.ts
  • x-pack/plugins/lists/tsconfig.json
  • x-pack/plugins/security_solution/server/lib/detection_engine/rule_exceptions/api/create_rule_exceptions/route.ts
  • x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/exception_lists_items/trial_license_complete_tier/lists/find_exception_lists.ts

elastic/security-detection-rule-management

  • packages/kbn-openapi-common/shared/index.ts
  • packages/kbn-openapi-common/shared/path_params_replacer.ts
  • packages/kbn-openapi-generator/src/template_service/templates/api_client_supertest.handlebars

elastic/security-solution

  • x-pack/plugins/security_solution/server/lib/detection_engine/rule_exceptions/api/create_rule_exceptions/route.ts
  • x-pack/plugins/security_solution/server/lib/exceptions/api/manage_exceptions/route.ts
  • x-pack/plugins/security_solution/tsconfig.json
  • x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/exception_lists_items/trial_license_complete_tier/lists/find_exception_lists.ts
  • x-pack/test/security_solution_api_integration/test_suites/security_solution_endpoint_api_int/apis/endpoint_artifacts/blocklists.ts

Copy link
Contributor

@paul-tavares paul-tavares left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes tagged for elastic/security-defend-workflows look good 👍

@yctercero yctercero requested a review from e40pud June 26, 2024 15:54
@maximpn maximpn force-pushed the add-missing-exceptions-api-oas branch from 5268734 to f64d0b8 Compare June 28, 2024 10:05
@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
lists 291 290 -1
securitySolution 5538 5537 -1
total -2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/securitysolution-io-ts-list-types 525 521 -4
Unknown metric groups

API count

id before after diff
@kbn/securitysolution-io-ts-list-types 538 534 -4

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @maximpn

Copy link
Contributor

@e40pud e40pud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Left a few comments/questions - mostly regarding the old vs new schemas and whether we do breaking changes in some places.

export const ExceptionListTypeEnum = ExceptionListType.enum;

export type ExceptionListName = z.infer<typeof ExceptionListName>;
export const ExceptionListName = NonEmptyString;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non empty string is not required by the old schema packages/kbn-securitysolution-io-ts-list-types/src/common/name/index.ts, so this will be a breaking change.

export const name = t.string;
export type Name = t.TypeOf<typeof name>;


export type CreateExceptionListRequestBody = z.infer<typeof CreateExceptionListRequestBody>;
export const CreateExceptionListRequestBody = z.object({
list_id: ExceptionListHumanId.optional(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the old schema, we would check if the input is null and create a uuid in that case, otherwise we would expect non empty string. I do not see the parity in behaviour here. Is it possible to do the same via openapi and zod?

/**
 * Types the DefaultUuid as:
 *   - If null or undefined, then a default string uuidv4() will be
 *     created otherwise it will be checked just against an empty string
 */
export const DefaultUuid = new t.Type<string, string | undefined, unknown>(
  'DefaultUuid',
  t.string.is,
  (input, context): Either<t.Errors, string> =>
    input == null ? t.success(uuidv4()) : NonEmptyString.validate(input, context),
  t.identity
);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add .default(uuidv4())?

export const ExceptionNamespaceTypeEnum = ExceptionNamespaceType.enum;

export type ExceptionListTags = z.infer<typeof ExceptionListTags>;
export const ExceptionListTags = z.array(NonEmptyString);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see the requirement for non empty string for this type in old version.

export const tags = DefaultStringArray;

where

export const DefaultStringArray = new t.Type<string[], string[] | undefined, unknown>(
  'DefaultStringArray',
  t.array(t.string).is,
  (input, context): Either<t.Errors, string[]> =>
    input == null ? t.success([]) : t.array(t.string).validate(input, context),
  t.identity
);

export const ExceptionListOsTypeArray = z.array(ExceptionListOsType);

export type ExceptionListVersion = z.infer<typeof ExceptionListVersion>;
export const ExceptionListVersion = z.number().int().min(1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In old schema we use PositiveIntegerGreaterThanZero type for the version. Should we add check for that as well?

export type CreateExceptionListItemRequestBody = z.infer<typeof CreateExceptionListItemRequestBody>;
export const CreateExceptionListItemRequestBody = z.object({
item_id: ExceptionListItemHumanId.optional(),
list_id: ExceptionListHumanId,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as in case with exception list schema, do we need to add .default(uuidv4()) here?

- `exception-list-agnostic`: Specify an exception list that is shared across spaces.

*/
filter: FindExceptionListsFilter.optional(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in packages/kbn-securitysolution-io-ts-list-types/src/common/filter/index.ts non empty string for filter is not required

/**
* The page number to return
*/
page: z.coerce.number().int().min(0).optional(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to packages/kbn-securitysolution-io-ts-types/src/string_to_positive_number/index.ts both page and per_page should be positive numbers. We use StringToPositiveNumber type which does failure check stringAsNumber <= 0. Can we use z.number().positive() here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same in other files where we use these types.

/**
* Determines which field is used to sort the results
*/
sort_field: NonEmptyString.optional(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Old schema does not require non empty string for sort_field

export type FindExceptionListsResponse = z.infer<typeof FindExceptionListsResponse>;
export const FindExceptionListsResponse = z.object({
data: z.array(ExceptionList),
page: z.number().int().min(0),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three page, per_page and total marked with // TODO: Change this out for PositiveNumber from siem. Maybe we should add .positive() to these types instead of .min(0).

}

return response.ok({ body: DeleteExceptionListItemResponse.parse(deleted) });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that before we would return 500 error if response validation fails. Is this still the same in case of zod parse?

@yctercero
Copy link
Contributor

@maximpn do these changes require going in 8.15 or can it wait for 8.16?

cc @banderror

@banderror banderror added v8.16.0 and removed v8.15.0 labels Jul 1, 2024
@banderror
Copy link
Contributor

@yctercero We don't have to merge this before the FF - we can take our time to review this. I updated the labels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs release_note:skip Skip the PR/issue when compiling release notes Team:Detection Engine Security Solution Detection Engine Area Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants