[Security Solutions] Adds back the legacy actions and notification system in a limited fashion

[FrankHassanabad]

Summary

Fixes https://github.com/elastic/security-team/issues/1759

Related earlier PR, #109722, where these were removed to where they could no longer function. This PR adds them back to where they will function for existing users. The end goal is to have users naturally migrate as they update, enable/disable, or create new rules.

What this PR does:

• Adds back the legacy side car actions siem-detection-engine-rule-actions
• Adds back the legacy hidden alert of siem.notifications
• Adds back unit tests where they existed. Both of these systems did not have existing e2e tests.
• Re-adds the find feature and functionality which should show the rules with legacy and non-legacy notifications/side car actions during a REST find operation.
• Updates the logic for when to show a legacy vs. non-legacy notification/side car action.
• Adds a new route called /internal/api/detection/legacy/notifications which is only for developer and tests for us to maintain this system for the foreseeable future.
• Adds script to exercise creating old notifications detection_engine/scripts/post_legacy_notification.sh
• Adds a data file for the script to use as an example for ad-hoc testing, scripts/legacy_notifications/one_action.json
• Adds within security_solution/server/types.ts ActionsApiRequestHandlerContext so that if we need to directly access actions within plugins we can. I do not use it here, but it should have been existing there and is good to have it in case we need it at this point within REST routes.
• When adding back the files and changes, I use the kibana-core approach of prefixing files, functions, types, etc... with the words legacyFoo. The files are named legacy_foo.ts. Everything has @deprecation above them as well. The intent here is all of this should hopefully make it unambiguously clear which parts of the notification system are for the new system/existing API and which ones are only for the deprecated legacy system. There exists some parts of the system that are used within both and the hope is that we can keep the legacy pieces separate from the non-legacy pieces for strangling the legacy pieces.
• This adds a new linter rule to prevent users from easily importing files named legacy_foo.ts or foo_legacy.ts we are using here and can also use for other similar legacy parts of the system we have. This seems to be the established pattern that kibana-core does as well looking through the linters and code base.
• Removes some dead import/export code and types instead of maintaining them since they are no longer used.

What this PR does not do (but are planned on follow ups):

• This PR does not add migration logic in most conditions such as a user enabling/disabling a rule, editing a rule unless the user is explicitly changing the actions by turning off the notification and then re-adding the notification.
• This PR does not log any information indicating to the user that they are running legacy rules or indicates they have that.
• This PR does not allow the executors or any UI/UX, backend to re-add a legacy notification. Instead only the hidden REST route of /internal/api/detection/legacy/notifications allows us to do this for testing purposes.
• This PR does not migrate the data structure of actions legacy notification system siem-detection-engine-rule-actions to use saved object references.
• If you delete an alert this will not delete the side car if it detects one is present on it.
• If you update an alert notification with a new notification this will not remove the side car on the update.

Ad-hoc testing instructions
How to do ad-hoc testing for various situations such as having a legacy notification system such as a user's or if you want to mimic a malfunction and result of a "split-brain" to where you have both notification systems running at the same time due to a bug or regression:

Create a rule and activate it normally within security_solution:

Do not add actions to the rule at this point as we will first exercise the older legacy system. However, you want at least one action configured such as a slack notification:

Within dev tools do a query for all your actions and grab one of the _id of them without their prefix:

# See all your actions
GET .kibana/_search
{
  "query": {
    "term": {
      "type": "action"
    }
  }
}

Mine was "_id" : "action:879e8ff0-1be1-11ec-a722-83da1c22a481", so I will be copying the ID of 879e8ff0-1be1-11ec-a722-83da1c22a481

Go to the file detection_engine/scripts/legacy_notifications/one_action.json and add this id to the file. Something like this:

{
  "name": "Legacy notification with one action",
  "interval": "1m",  <--- You can use whatever you want. Real values are "1h", "1d", "1w". I use "1m" for testing purposes.
  "actions": [
    {
      "id": "879e8ff0-1be1-11ec-a722-83da1c22a481", <--- My action id
      "group": "default",
      "params": {
        "message": "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
      },
      "actionTypeId": ".slack" <--- I am a slack action id type.
    }
  ]
}

Query for an alert you want to add manually add back a legacy notification to it. Such as:

# See all your siem.signals alert types and choose one
GET .kibana/_search
{
  "query": {
    "term": {
      "alert.alertTypeId": "siem.signals"
    }
  }
}

Grab the _id without the alert prefix. For mine this was 933ca720-1be1-11ec-a722-83da1c22a481

Within the directory of detection_engine/scripts execute the script

./post_legacy_notification.sh 933ca720-1be1-11ec-a722-83da1c22a481
{
  "ok": "acknowledged"
}

which is going to do a few things. See the file detection_engine/routes/rules/legacy_create_legacy_notification.ts for the definition of the route and what it does in full, but we should notice that we have now:

Created a legacy side car action object of type siem-detection-engine-rule-actions you can see in dev tools:

# See the actions "side car" which are part of the legacy notification system.
GET .kibana/_search
{
  "query": {
    "term": {
      "type": {
        "value": "siem-detection-engine-rule-actions"
      }
    }
  }
}

Note in the response:

          "siem-detection-engine-rule-actions" : {
            "ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481", <--- NOTE, not migrated to references yet
            "actions" : [
              {
                "action_type_id" : ".slack",
                "id" : "879e8ff0-1be1-11ec-a722-83da1c22a481", <--- NOTE, not migrated to references yet
                "params" : {
                  "message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
                },
                "group" : "default"
              }
            ],
            "ruleThrottle" : "1m", <--- Should be the same as the interval in "one_action.json" config
            "alertThrottle" : "1m" <--- Should be the same as the interval in "one_action.json" config
          },
          "type" : "siem-detection-engine-rule-actions",
          "references" : [ ],

Created a siem.notification rule instance which you can see in dev tools as well:

# Get the alert type of "siem-notifications" which is part of the legacy system.
GET .kibana/_search
{
  "query": {
    "term": {
      "alert.alertTypeId": "siem.notifications"
    }
  }
}

Take note from the siem.notifications these values which determine how/when it fires and if your actions are set up correctly:

            "name" : "Legacy notification with one action" <--- Our name from one_action.json 
            "schedule" : {
              "interval" : "1m" <--- Interval should match interval in one_action.json
            },
            "enabled" : true, <--- We should be enabled
            "actions" : [
              {
                "group" : "default",
                "params" : {
                  "message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
                },
                "actionTypeId" : ".slack", <--- Our actionID
                "actionRef" : "action_0"
              }
            ],

And that now there exists a task within task manager that will be executing this:

# Get the tasks of siem notifications to ensure and see it is running
GET .task-manager/_search
{
  "query": {
    "term": {
      "task.taskType": "alerting:siem.notifications"
    }
  }
}

You can double check the interval from the result of the query to ensure it runs as the configuration test file shows it should be:

            "schedule" : {
              "interval" : "1m"
            },

Within time you should see your action execute like the legacy notification system:

If you go to edit the rule you should notice that the rule now has the side car attached to it within the UI:

You can also look at your log messages in debug mode to verify the behaviors of the legacy system and the normal rules running.

Compare these data structures to a 7.14.x system in cloud to ensure the data looks the same and the ad-hoc testing functions as expected.

Check the scripts of ./find_rules.sh, ./read_rules.sh to ensure that the find REST route returns the legacy actions when they are there.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[FrankHassanabad]

NOTE: These are "extra's" and not needed for this PR but I figured since I added 'em I should keep 'em since we have them in the plugins and we might use them later on.

[yctercero]

Thanks so much for working to add this back in, making it work with the new actions as well. Did a good amount of testing together over zoom, but also went ahead and tested manually later as well.

Tested actions in 7.14 to understand the legacy system and any existing oddities. Tested this PR with new actions.

Big thanks to @dhurley14 who tested the migration itself. There was one thing that Devin noted that I think is worth writing an issue to follow up with alerting on.

[dhurley14]

One comment about expanding on when exactly we can remove references to the legacy functions as a part of their @deprecated message. Another comment about ensuring error handling is accounted for when attempting to acquire the actions as part of the find_rules_route. Other than that, thank you @FrankHassanabad for taking the time to walk me through this and helping facilitate testing this functionality. LGTM!

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request
• Commit: cdf4d81
• Storybooks Preview
• Documentation Changes
• Flaky suites:
  • xpack-kibana-ciGroup13

---

Test Failures

Kibana Pipeline / general / License plugin API Integration Tests.x-pack/test/licensing_plugin/server/updates·ts.Licensing plugin server client changes in license types provides changes in license types

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://github.com/elastic/kibana/issues/110938

[00:00:00]                                             │
[00:00:00]                                               └-: Licensing plugin server client
[00:00:00]                                                 └-> "before all" hook in "Licensing plugin server client"
[00:00:00]                                                 └-: changes in license types
[00:00:00]                                                   └-> "before all" hook for "provides changes in license types"
[00:00:00]                                                   └-> provides changes in license types
[00:00:00]                                                     └-> "before each" hook: global before each for "provides changes in license types"
[00:00:00]                                                     │ debg creating role license_manager-role
[00:00:00]                                                     │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [license_manager-role]
[00:00:00]                                                     │ debg creating user license_manager_user
[00:00:00]                                                     │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [license_manager_user]
[00:00:00]                                                     │ debg created user license_manager_user
[00:00:00]                                                     │ debg SecurityPage.logout
[00:00:00]                                                     │ debg TestSubjects.exists(userMenuButton)
[00:00:00]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenuButton"]') with timeout=2500
[00:00:03]                                                     │ debg --- retry.tryForTime error: [data-test-subj="userMenuButton"] is not displayed
[00:00:03]                                                     │ debg Logout not found
[00:00:03]                                                     │ debg TestSubjects.exists(loginForm)
[00:00:03]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="loginForm"]') with timeout=2500
[00:00:06]                                                     │ debg --- retry.tryForTime error: [data-test-subj="loginForm"] is not displayed
[00:00:06]                                                     │ debg navigating to login url: http://localhost:61231/login
[00:00:06]                                                     │ debg navigate to: http://localhost:61231/login
[00:00:07]                                                     │ debg browser[INFO] http://localhost:61231/login?_t=1632779845971 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:07]                                                     │
[00:00:07]                                                     │ debg browser[INFO] http://localhost:61231/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:07]                                                     │ debg ... sleep(700) start
[00:00:07]                                                     │ debg ... sleep(700) end
[00:00:07]                                                     │ debg returned from get, calling refresh
[00:00:09]                                                     │ debg browser[INFO] http://localhost:61231/login?_t=1632779845971 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:09]                                                     │
[00:00:09]                                                     │ debg browser[INFO] http://localhost:61231/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:09]                                                     │ debg currentUrl = http://localhost:61231/login
[00:00:09]                                                     │          appUrl = http://localhost:61231/login
[00:00:09]                                                     │ debg TestSubjects.find(kibanaChrome)
[00:00:09]                                                     │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:11]                                                     │ debg ... sleep(501) start
[00:00:11]                                                     │ERROR browser[SEVERE] http://localhost:61231/api/licensing/info - Failed to load resource: the server responded with a status of 401 (Unauthorized)
[00:00:11]                                                     │ debg ... sleep(501) end
[00:00:11]                                                     │ debg in navigateTo url = http://localhost:61231/login
[00:00:11]                                                     │ debg Waiting for Login Form to appear.
[00:00:11]                                                     │ debg Waiting up to 100000ms for login form...
[00:00:11]                                                     │ debg TestSubjects.exists(loginForm)
[00:00:11]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="loginForm"]') with timeout=2500
[00:00:11]                                                     │ debg TestSubjects.setValue(loginUsername, license_manager_user)
[00:00:11]                                                     │ debg TestSubjects.click(loginUsername)
[00:00:11]                                                     │ debg Find.clickByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:11]                                                     │ debg Find.findByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:11]                                                     │ debg TestSubjects.setValue(loginPassword, license_manager_user-password)
[00:00:11]                                                     │ debg TestSubjects.click(loginPassword)
[00:00:11]                                                     │ debg Find.clickByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:11]                                                     │ debg Find.findByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:12]                                                     │ debg TestSubjects.click(loginSubmit)
[00:00:12]                                                     │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:12]                                                     │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:12]                                                     │ debg Waiting for login result, expected: undefined.
[00:00:12]                                                     │ debg Waiting up to 20000ms for logout button visible...
[00:00:12]                                                     │ debg TestSubjects.exists(userMenuButton)
[00:00:12]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenuButton"]') with timeout=2500
[00:00:12]                                                     │ proc [kibana]   log   [21:57:31.361] [info][plugins][routes][security] Logging in with provider "basic" (basic)
[00:00:14]                                                     │ debg --- retry.tryForTime error: [data-test-subj="userMenuButton"] is not displayed
[00:00:15]                                                     │ debg browser[INFO] http://localhost:61231/app/home 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:15]                                                     │
[00:00:15]                                                     │ debg browser[INFO] http://localhost:61231/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:15]                                                     │ debg TestSubjects.exists(userMenuButton)
[00:00:15]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenuButton"]') with timeout=2500
[00:00:17]                                                     │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/gy_tfBOaSOyvKY_R2cvW1A] update_mapping [_doc]
[00:00:17]                                                     │ info [o.e.x.i.IndexLifecycleTransition] [node-01] moving index [.ds-ilm-history-5-2021.09.27-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] in policy [ilm-history-ilm-policy]
[00:00:17]                                                     │ info [o.e.x.i.IndexLifecycleTransition] [node-01] moving index [.ds-ilm-history-5-2021.09.27-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [ilm-history-ilm-policy]
[00:00:17]                                                     │ debg TestSubjects.exists(userMenu)
[00:00:17]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenu"]') with timeout=2500
[00:00:20]                                                     │ debg --- retry.tryForTime error: [data-test-subj="userMenu"] is not displayed
[00:00:20]                                                     │ debg TestSubjects.click(userMenuButton)
[00:00:20]                                                     │ debg Find.clickByCssSelector('[data-test-subj="userMenuButton"]') with timeout=10000
[00:00:20]                                                     │ debg Find.findByCssSelector('[data-test-subj="userMenuButton"]') with timeout=10000
[00:00:20]                                                     │ debg TestSubjects.exists(userMenu)
[00:00:20]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenu"]') with timeout=120000
[00:00:20]                                                     │ debg TestSubjects.exists(userMenu > logoutLink)
[00:00:20]                                                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenu"] [data-test-subj="logoutLink"]') with timeout=2500
[00:00:22]                                                     │ info [o.e.l.LicenseService] [node-01] license [5247cc5e-6712-44ed-a488-10fd40662757] mode [trial] - valid
[00:00:22]                                                     │ info [o.e.x.s.a.Realms] [node-01] license mode is [trial], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[00:00:23]                                                     │ proc [kibana]   log   [21:57:42.728] [info][kibana-monitoring][monitoring][monitoring][plugins] Starting monitoring stats collection
[00:00:23]                                                     │ info Taking screenshot "/dev/shm/workspace/parallel/23/kibana/x-pack/test/licensing_plugin/screenshots/failure/Licensing plugin server client changes in license types provides changes in license types.png"
[00:00:24]                                                     │ info Current URL is: http://localhost:61231/app/home#/
[00:00:24]                                                     │ info Saving page source to: /dev/shm/workspace/parallel/23/kibana/x-pack/test/licensing_plugin/failure_debug/html/Licensing plugin server client changes in license types provides changes in license types.html
[00:00:24]                                                     └- ✖ fail: Licensing plugin server client changes in license types provides changes in license types
[00:00:24]                                                     │      Error: expected 'basic' to equal 'trial'
[00:00:24]                                                     │       at Assertion.assert (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:00:24]                                                     │       at Assertion.be.Assertion.equal (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:227:8)
[00:00:24]                                                     │       at Assertion.be (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:69:22)
[00:00:24]                                                     │       at Context.<anonymous> (test/licensing_plugin/server/updates.ts:45:45)
[00:00:24]                                                     │       at runMicrotasks (<anonymous>)
[00:00:24]                                                     │       at processTicksAndRejections (internal/process/task_queues.js:95:5)
[00:00:24]                                                     │       at Object.apply (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:00:24]                                                     │ 
[00:00:24]                                                     │ 

Stack Trace

Error: expected 'basic' to equal 'trial'
    at Assertion.assert (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:227:8)
    at Assertion.be (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:69:22)
    at Context.<anonymous> (test/licensing_plugin/server/updates.ts:45:45)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at Object.apply (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

---

Metrics [docs]

✅ unchanged

History

• 💛 Build #155877 was flaky db725c0
• 💚 Build #155775 succeeded a1a94c5
• 💚 Build #155573 succeeded 9dbaae6
• 💔 Build #155526 failed 14e9394
• 💔 Build #155517 failed 3ec5a56

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @FrankHassanabad

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.