Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.x] [SIEM][Detection Engine] Rule Status Monitoring (#54452) #54716

Merged
merged 1 commit into from Jan 14, 2020

Conversation

@dhurley14
Copy link
Contributor

dhurley14 commented Jan 14, 2020

Backports the following commits to 7.x:

  • [SIEM][Detection Engine] Rule Status Monitoring (#54452)
* Working status updates in executor. Need to update read rules api endpoint to only respond with 'status' and not status info. Will create another endpoint to get status details for a rule which will include last five errors (if there are any). Still need tests

* adds new route for getting statuses for a list of given alert ids, adds try-catch and more logic in executor for logging errors, adds scripts and rules for testing, updates find_rules endpoint to display statuses too. Would like to look into using the alerts executor state to better manage logic for statuses, and need to update some types. Also needs unit tests still.

* updated types for routes, updated how merging of alert-to-rule and rule status happens when formatting REST response.

* typecast test server as ServerFacade type

* fix bug where we were not awaiting the accumulated result in the reducer

* update rule status saved object interfaces to play nicely with interfaces provided by saved objects module. Update tests to pass - Need to write new unit tests in an upcoming commit. Next commit will be cleanup from comments then new unit tests.

* fix missed conflicts after rebase

* replace id param with rule.id when searching in statuses, adds sort fields to the saved objects find queries.

* fixes bug where 'executing' statuses were being written into failing historical status list

* camelCase to snake_case in new statuses route, also fix merge conflict

* add deletion of rule statuses to delete_rules_bulk_route. Statuses are created inside of executor so we will not be needing to create statuses directly inside of the create rules bulk route, so I removed that extraneous code.

* pr feedback I forgot to fix earlier

* remove unused import. fixes type check error generated in previous commit

* removes status information from rule when saved to signals index and updates tests to represent this change. Also removes extraneous quotes inserted around alertId field when creating a new historical status.

* adds new bash script to delete all rule statuses, updates error messages in rule statuses to just store actual message, moved querying of rules statuses under a null check, initialize everything to null when first creating rule status, update number of results returned when querying saved objects based on usage, updates saved objects mapping types to use date for dates and keyword for alertId.

* use lodash snake case and update total number of saved objects to return for find rules, delete rules, and read rules.

* updates how statuses are transformed inside of read_rules_route, only update updated_at in rule on update of rule, removes unlabeled todo comment, updates scripts descriptions, removes interval from query_with_rule_id.json sample query, removes debug statement, removes verbose from curl script.

* display rule status on update
@dhurley14 dhurley14 added the backport label Jan 14, 2020
@kibanamachine

This comment has been minimized.

Copy link

kibanamachine commented Jan 14, 2020

💚 Build Succeeded

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@dhurley14 dhurley14 merged commit 68ff6f6 into elastic:7.x Jan 14, 2020
55 checks passed
55 checks passed
API integration tests node scripts/functional_tests --config test/api_integration/config.js --bail --debug
Details
Browser tests yarn run grunt test:browser-ci
Details
Build kbn_tp_sample_panel_action yarn build
Details
CLA All commits in pull request signed
Details
Check core API changes node scripts/check_core_api_changes
Details
Check file casing node scripts/check_file_casing --quiet
Details
Check licenses node scripts/check_licenses --dev
Details
Example functional tests node scripts/functional_tests --config test/examples/config.js --bail --debug
Details
Firefox smoke test node scripts/functional_tests --bail --debug --kibana-install-dir /dev/shm/workspace/kibana/build/oss/kibana-7.6.0-SNAPSHOT-linux-x86_64 --include-tag smoke --config test/functional/config.firefox.js
Details
Functional tests / Group 1 yarn run grunt run:functionalTests_ciGroup1
Details
Functional tests / Group 10 yarn run grunt run:functionalTests_ciGroup10
Details
Functional tests / Group 11 yarn run grunt run:functionalTests_ciGroup11
Details
Functional tests / Group 12 yarn run grunt run:functionalTests_ciGroup12
Details
Functional tests / Group 2 yarn run grunt run:functionalTests_ciGroup2
Details
Functional tests / Group 3 yarn run grunt run:functionalTests_ciGroup3
Details
Functional tests / Group 4 yarn run grunt run:functionalTests_ciGroup4
Details
Functional tests / Group 5 yarn run grunt run:functionalTests_ciGroup5
Details
Functional tests / Group 6 yarn run grunt run:functionalTests_ciGroup6
Details
Functional tests / Group 7 yarn run grunt run:functionalTests_ciGroup7
Details
Functional tests / Group 8 yarn run grunt run:functionalTests_ciGroup8
Details
Functional tests / Group 9 yarn run grunt run:functionalTests_ciGroup9
Details
Internationalization check node scripts/i18n_check --ignore-missing
Details
Interpreter functional tests node scripts/functional_tests --config test/interpreter_functional/config.ts --bail --debug --kibana-install-dir /dev/shm/workspace/kibana/build/oss/kibana-7.6.0-SNAPSHOT-linux-x86_64-2
Details
Jest integration tests yarn run grunt test:jest_integration
Details
Jest tests yarn run grunt test:jest
Details
Kibana accessibility tests node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/kibana/build/oss/kibana-7.6.0-SNAPSHOT-linux-x86_64 --config test/accessibility/config.ts
Details
Mocha tests node scripts/mocha
Details
Plugin functional tests node scripts/functional_tests --config test/plugin_functional/config.js --bail --debug
Details
Project tests yarn run grunt test:projects
Details
Type check node scripts/type_check
Details
TypeScript - all files belong to a TypeScript project node scripts/check_ts_projects
Details
Verify NOTICE.txt node scripts/notice --validate
Details
Verify dependency versions yarn run grunt verifyDependencyVersions
Details
X-Pack Chrome Functional tests / Group 1 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-2 --include-tag ciGroup1
Details
X-Pack Chrome Functional tests / Group 10 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-11 --include-tag ciGroup10
Details
X-Pack Chrome Functional tests / Group 2 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-3 --include-tag ciGroup2
Details
X-Pack Chrome Functional tests / Group 3 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-4 --include-tag ciGroup3
Details
X-Pack Chrome Functional tests / Group 4 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-5 --include-tag ciGroup4
Details
X-Pack Chrome Functional tests / Group 5 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-6 --include-tag ciGroup5
Details
X-Pack Chrome Functional tests / Group 6 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-7 --include-tag ciGroup6
Details
X-Pack Chrome Functional tests / Group 7 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-8 --include-tag ciGroup7
Details
X-Pack Chrome Functional tests / Group 8 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-9 --include-tag ciGroup8
Details
X-Pack Chrome Functional tests / Group 9 node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-10 --include-tag ciGroup9
Details
X-Pack Jest node scripts/jest --ci --verbose
Details
X-Pack Karma Tests yarn test:browser
Details
X-Pack SIEM cyclic dependency test node legacy/plugins/siem/scripts/check_circular_deps
Details
X-Pack accessibility tests node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana --config test/accessibility/config.ts
Details
X-Pack firefox smoke test node scripts/functional_tests --debug --bail --kibana-install-dir /dev/shm/workspace/install/kibana-1 --include-tag smoke --config test/functional/config.firefox.js
Details
elasticsearch-ci/docs Build finished.
Details
eslint node scripts/eslint --no-cache
Details
kibana-ci Build finished.
Details
prbot:outdated
prbot:release note labels
prbot:release version labels
sasslint node scripts/sasslint
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.