[Security Solution] [Detections] Adds support for system actions (and cases action) to detection rules

[dhurley14]

Summary

Usually, connectors are used to communicate with external services like PagerDuty or ServiceNow. Automated actions like opening a case or running an OsQuery do not need any configuration. These types of actions are known as "system actions" and represent ways to automate functionality between different kibana features.

This PR allows detection rules to take advantage of system actions, like the cases action, to expedite tasks which our security customers might already undertake, while providing support for future system actions to be added later. You can read more about the cases action here: #153837

Testing

The cases system action is only available for Platinum licensing. To test all we need is to create a rule and in the Actions tab, select Cases, then turn the rule on and have it generate alerts. When the alerts are generated, a case (or cases) will be created and can be viewed in the Cases page.

The configuration options shown in the above screenshot are explained in more detail in this RFC, specifically Diagram 1, but to summarize:

1. Group by field determines how the alerts are attached to a specific or other cases
2. Time window is for whenever a case is older than the given time window, and the rule finds more alerts, the case system action will generate a new case after the "time window" period has elapsed.
3. Reopen - This will reopen the case if the rule discovers alerts after the case has been closed.

The grouping pieces can be better understood via the following example:

If I select agent.type in the group by field for the cases action, and have a rule that generates alerts from packetbeat-* and from auditbeat-* then that singular rule will generate two separate cases. One for alerts grouped by agent.type: packetbeat and another case for auditbeat

And we can still use system actions in conjunction with other action connectors.

Checklist

Delete any items that are not applicable to this PR.

• Documentation was added for features that require explanation or tutorials: Security Solution docs ticket here
• Unit or functional tests were updated or added to match the most common scenarios (Todo)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Impact of failing system action on other actions associated with the rule	Low	High	Ensure testing system actions and third party connectors, mock a failure in cases api endpoints

[dhurley14]

/ci

[dhurley14]

For any questions on this, I pulled this from the system actions RFC:

All rules APIs will not accept the systemActions in their requests. The system will deduct if an action is a system action automatically using the actions’ client. As system actions are not allowed to have a group, the property will be optional (at the moment it is required) in all APIs. The APIs will convert the actions to the schema needed by the rules client. Finally, the APIs will merge the system actions with the default actions before sending the response back to the client.

Ref: https://docs.google.com/document/d/1mYStNLxrafnyxR6v_uB4ONw_iFHTg1pmE3EAQYgKFSY/edit?pli=1

[dhurley14]

/ci

[elasticmachine]

⏳ Build in-progress

• Buildkite Build
• Commit: 604a755
• Cloud Deployment
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-183937-604a755628d7
• Security Deployment

History

• 💚 Build #222247 succeeded 374309c
• 💛 Build #222212 was flaky 374309c
• 💔 Build #222186 failed 991e10a
• 💔 Build #221920 failed cc33a01

cc @dhurley14

[dhurley14]

Here is a list of test cases I tried manually. All passed:

1. Can I update a rule’s query without modifying or deleting system action? ✅
2. Can I update a single rule’s system action?✅
3. Can I duplicate a single rule and have the system action be present?✅
4. Can I update a single rule’s action without modifying / deleting the system action?✅
5. Can I create a rule with two non-system actions?✅
6. Can I create a rule with two non-system actions and the cases action?✅
7. Can I update a rule's non-system action?✅
8. Can I use the bulk add action feature to add a system action?✅
9. Can I use the bulk add action feature to add a non-system action and ensure it did not remove a system action already present on a rule?✅
10. Can I use the bulk add action feature to add the cases system action and ensure it did not remove a non-system action from a rule?✅
11. Can I use the bulk add action feature to add the cases action and overwrite other rules actions?✅
12. Can I use the bulk add action feature to add a non-system action and overwrite the other rules actions, including overwriting a system action?✅
13. Can I import and export a rule with system actions?✅
14. Will a case be created if an alert does not have a field?✅ (displays 'unknown' as the group by value and attaches the correct number of alerts)

[jpdjere]

Hey @dhurley14. Thanks for addressing my points. Rechecked the bulk adding of system actions and everything seems to be working as expected now 👍

Left a couple of nits, but approving ✅

(I did cause a conflict in x-pack/plugins/security_solution/server/lib/detection_engine/rule_management/logic/detection_rules_client/converters/internal_rule_to_api_response.ts when I merged #188631, sorry about that)

[vitaliidm]

Thanks for implementing this feature and addressing comments.

Approving it, only one test to add left: #183937 (comment)

[kibana-ci]

⏳ Build in-progress

• Buildkite Build
• Commit: 739a6a3
• Cloud Deployment
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-183937-739a6a3bb22b
• Security Deployment

History

• 💔 Build #223647 failed 8d517b4
• 💔 Build #223520 failed 8de96fa
• 💚 Build #222723 succeeded 09716a2
• 💛 Build #219138 was flaky 9eeb2e6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @dhurley14