[Security Solution][Detections] Migrate to using Saved Object References array to reference Exception Lists from Rules #101975
Assignees
Labels
enhancement
New value added to drive a business result
Feature:Rule Exceptions
Security Solution Rule Exceptions feature
fixed
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Comments
spong
added
enhancement
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
3 tasks
FrankHassanabad
added a commit
that referenced
this issue
Aug 11, 2021
…ved object references when created or modified (part 1) (#107064) ## Summary This is part 1 to addressing the issue seen here: #101975 This part 1 wires up our rules to be able to `inject` and `extract` parameters from the saved object references. Follow up part 2 (not included here) will do the saved object migrations of existing rules to have the saved object references. The way the code is written it shouldn't interfere or blow up anything even though the existing rules have not been migrated since we do fallbacks and only log errors when we detect that the saved object references have not been migrated or have been deleted. Therefore this PR should be migration friendly in that you will only see an occasional error as it serializes and deserializes a non migrated rule without object references but still work both ways. Non-migrated rules or rules with deleted saved object references will self correct during the serialization phase when you edit a rule and save out the modification. This should be migration bug friendly as well in case something does not work out with migrations, we can still have users edit an existing rule to correct the bug. For manual testing, see the `README.md` in the folder. You should be able to create and modify existing rules and then see in their saved objects that they have `references` pointing to the top level exception list containers with this PR. * Adds the new folder in `detection_engine/signals/saved_object_references` with all the code needed * Adds a top level `README.md` about the functionality and tips for new programmers to add their own references * Adds a generic pattern for adding more saved object references within our rule set * Adds ~40 unit tests * Adds additional migration safe logic to de-couple this from required saved object migrations and hopefully helps mitigates any existing bugs within the stack or previous migration bugs a bit for us. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Aug 11, 2021
…ved object references when created or modified (part 1) (elastic#107064) ## Summary This is part 1 to addressing the issue seen here: elastic#101975 This part 1 wires up our rules to be able to `inject` and `extract` parameters from the saved object references. Follow up part 2 (not included here) will do the saved object migrations of existing rules to have the saved object references. The way the code is written it shouldn't interfere or blow up anything even though the existing rules have not been migrated since we do fallbacks and only log errors when we detect that the saved object references have not been migrated or have been deleted. Therefore this PR should be migration friendly in that you will only see an occasional error as it serializes and deserializes a non migrated rule without object references but still work both ways. Non-migrated rules or rules with deleted saved object references will self correct during the serialization phase when you edit a rule and save out the modification. This should be migration bug friendly as well in case something does not work out with migrations, we can still have users edit an existing rule to correct the bug. For manual testing, see the `README.md` in the folder. You should be able to create and modify existing rules and then see in their saved objects that they have `references` pointing to the top level exception list containers with this PR. * Adds the new folder in `detection_engine/signals/saved_object_references` with all the code needed * Adds a top level `README.md` about the functionality and tips for new programmers to add their own references * Adds a generic pattern for adding more saved object references within our rule set * Adds ~40 unit tests * Adds additional migration safe logic to de-couple this from required saved object migrations and hopefully helps mitigates any existing bugs within the stack or previous migration bugs a bit for us. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine
added a commit
that referenced
this issue
Aug 11, 2021
…ved object references when created or modified (part 1) (#107064) (#108225) ## Summary This is part 1 to addressing the issue seen here: #101975 This part 1 wires up our rules to be able to `inject` and `extract` parameters from the saved object references. Follow up part 2 (not included here) will do the saved object migrations of existing rules to have the saved object references. The way the code is written it shouldn't interfere or blow up anything even though the existing rules have not been migrated since we do fallbacks and only log errors when we detect that the saved object references have not been migrated or have been deleted. Therefore this PR should be migration friendly in that you will only see an occasional error as it serializes and deserializes a non migrated rule without object references but still work both ways. Non-migrated rules or rules with deleted saved object references will self correct during the serialization phase when you edit a rule and save out the modification. This should be migration bug friendly as well in case something does not work out with migrations, we can still have users edit an existing rule to correct the bug. For manual testing, see the `README.md` in the folder. You should be able to create and modify existing rules and then see in their saved objects that they have `references` pointing to the top level exception list containers with this PR. * Adds the new folder in `detection_engine/signals/saved_object_references` with all the code needed * Adds a top level `README.md` about the functionality and tips for new programmers to add their own references * Adds a generic pattern for adding more saved object references within our rule set * Adds ~40 unit tests * Adds additional migration safe logic to de-couple this from required saved object migrations and hopefully helps mitigates any existing bugs within the stack or previous migration bugs a bit for us. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: Frank Hassanabad <frank.hassanabad@elastic.co>
1 task
FrankHassanabad
added a commit
that referenced
this issue
Aug 17, 2021
…ved object references (Part 2) (#108291) ## Summary This is part 2 to addressing the issue seen here: #101975 (Part 1 #107064) This adds the alerting migration scripts and unit tests for exception list containers on Kibana startup for `7.15.0` We only migrate if we find these conditions and cases: - `exceptionLists` are an `array` and not `null`, `undefined`, or malformed data. - The exceptionList item is an `object` and its `id` is a `string` and not `null`, `undefined`, or malformed data - The existing references do not already have an exceptionItem reference already found within it. We migrate on the common use case - The saved object references do not exist but we have exceptionList items with the id's to create the saved object references, 👍 so we migrate. - The alert contains no exception list items, in which case we have nothing to migrate We do these additional (mis-use) cases and steps as well. These should _NOT_ be common things that happen but we safe guard for them here: - If the migration is run twice we are idempotent and do _NOT_ add duplicates list items or remove items. - If the migration was partially successful but re-run a second time, we only add what is missing. Again no duplicates or removed items should occur. - If the `exceptionLists` contains invalid data shape or not enough information to migrate, we filter it out and ignore it - If the saved object references already exists and contains a different or foreign value, we will retain the foreign reference(s) and still migrate. ## Manual testing There are unit tests but for any manual testing or verification you can do the following: Create a few alerts through the `security_solution` application with exception lists <img width="1775" alt="Screen Shot 2021-08-11 at 5 42 31 PM" src="https://user-images.githubusercontent.com/1151048/129117377-61b17fcf-ad01-4405-bbfe-42d97a6f7654.png"> Use the dev tools to de-migrate as well as to test end to end like so: ```json # First get an "_id" with an exceptions list like below. Mine I found was: "alert:38482620-ef1b-11eb-ad71-7de7959be71c": GET .kibana/_search { "query": { "terms": { "alert.alertTypeId": [ "siem.signals" ] } }, "size": 10000 } ``` With Kibana running downgrade and remove the references as a test: ```json # Set saved object array references as empty arrays and set our migration version to be 7.14.0 POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; ctx._source.references = [] """, "lang": "painless" } } # Double check the references is empty and the version is 7.14.0 GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` Reload the alert in the `security_solution` and notice you get these errors until you restart Kibana to cause a migration moving forward ```sh server log [17:35:16.914] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"} server log [17:35:16.914] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[] server log [17:35:16.915] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"} server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"} server log [17:35:16.940] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[] server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"} ``` Restart Kibana and you should no longer have errors in the Kibana console. If you do this query in dev tools ```json # Check that the `migrationVersion` is `7.15.0` and that we have a `references` array filled out with the correct structure GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` You should notice that you now have a `references` array filled out: ```json "references" : [ { "name" : "param:exceptionsList_0", "id" : "endpoint_list", "type" : "exception-list-agnostic" }, { "name" : "param:exceptionsList_1", "id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c", "type" : "exception-list" } ], "migrationVersion" : { "alert" : "7.15.0" } ``` For testing [idempotentence](https://en.wikipedia.org/wiki/Idempotence) Run just this to downgrade and restart Kibana and you should notice on a GET that we do not have anything extra in the references array: ```json # Set our migration version to be 7.14.0 only POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; """, "lang": "painless" } } # Double check the `references` is still there, and we do not get errors or changes to `references` after we restart Kibana GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` For testing foreign keys: ```json # Set saved object array references to foreign keys and set our migration version to be 7.14.0 POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; ctx._source.references = [["name" : "foreign", "id" : "123", "type" : "some-type"]]; """, "lang": "painless" } } ``` Restart, ensure no errors in Kibana console and do a get call to ensure we have the foreign mixed with valid values: ```json GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` Should return this data: ```json "type" : "alert", "references" : [ { "name" : "foreign", "id" : "123", "type" : "some-type" }, { "name" : "param:exceptionsList_0", "id" : "endpoint_list", "type" : "exception-list-agnostic" }, { "name" : "param:exceptionsList_1", "id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c", "type" : "exception-list" } ] ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Aug 17, 2021
…ved object references (Part 2) (elastic#108291) ## Summary This is part 2 to addressing the issue seen here: elastic#101975 (Part 1 elastic#107064) This adds the alerting migration scripts and unit tests for exception list containers on Kibana startup for `7.15.0` We only migrate if we find these conditions and cases: - `exceptionLists` are an `array` and not `null`, `undefined`, or malformed data. - The exceptionList item is an `object` and its `id` is a `string` and not `null`, `undefined`, or malformed data - The existing references do not already have an exceptionItem reference already found within it. We migrate on the common use case - The saved object references do not exist but we have exceptionList items with the id's to create the saved object references, 👍 so we migrate. - The alert contains no exception list items, in which case we have nothing to migrate We do these additional (mis-use) cases and steps as well. These should _NOT_ be common things that happen but we safe guard for them here: - If the migration is run twice we are idempotent and do _NOT_ add duplicates list items or remove items. - If the migration was partially successful but re-run a second time, we only add what is missing. Again no duplicates or removed items should occur. - If the `exceptionLists` contains invalid data shape or not enough information to migrate, we filter it out and ignore it - If the saved object references already exists and contains a different or foreign value, we will retain the foreign reference(s) and still migrate. ## Manual testing There are unit tests but for any manual testing or verification you can do the following: Create a few alerts through the `security_solution` application with exception lists <img width="1775" alt="Screen Shot 2021-08-11 at 5 42 31 PM" src="https://user-images.githubusercontent.com/1151048/129117377-61b17fcf-ad01-4405-bbfe-42d97a6f7654.png"> Use the dev tools to de-migrate as well as to test end to end like so: ```json # First get an "_id" with an exceptions list like below. Mine I found was: "alert:38482620-ef1b-11eb-ad71-7de7959be71c": GET .kibana/_search { "query": { "terms": { "alert.alertTypeId": [ "siem.signals" ] } }, "size": 10000 } ``` With Kibana running downgrade and remove the references as a test: ```json # Set saved object array references as empty arrays and set our migration version to be 7.14.0 POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; ctx._source.references = [] """, "lang": "painless" } } # Double check the references is empty and the version is 7.14.0 GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` Reload the alert in the `security_solution` and notice you get these errors until you restart Kibana to cause a migration moving forward ```sh server log [17:35:16.914] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"} server log [17:35:16.914] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[] server log [17:35:16.915] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"} server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"} server log [17:35:16.940] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[] server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"} ``` Restart Kibana and you should no longer have errors in the Kibana console. If you do this query in dev tools ```json # Check that the `migrationVersion` is `7.15.0` and that we have a `references` array filled out with the correct structure GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` You should notice that you now have a `references` array filled out: ```json "references" : [ { "name" : "param:exceptionsList_0", "id" : "endpoint_list", "type" : "exception-list-agnostic" }, { "name" : "param:exceptionsList_1", "id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c", "type" : "exception-list" } ], "migrationVersion" : { "alert" : "7.15.0" } ``` For testing [idempotentence](https://en.wikipedia.org/wiki/Idempotence) Run just this to downgrade and restart Kibana and you should notice on a GET that we do not have anything extra in the references array: ```json # Set our migration version to be 7.14.0 only POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; """, "lang": "painless" } } # Double check the `references` is still there, and we do not get errors or changes to `references` after we restart Kibana GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` For testing foreign keys: ```json # Set saved object array references to foreign keys and set our migration version to be 7.14.0 POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; ctx._source.references = [["name" : "foreign", "id" : "123", "type" : "some-type"]]; """, "lang": "painless" } } ``` Restart, ensure no errors in Kibana console and do a get call to ensure we have the foreign mixed with valid values: ```json GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` Should return this data: ```json "type" : "alert", "references" : [ { "name" : "foreign", "id" : "123", "type" : "some-type" }, { "name" : "param:exceptionsList_0", "id" : "endpoint_list", "type" : "exception-list-agnostic" }, { "name" : "param:exceptionsList_1", "id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c", "type" : "exception-list" } ] ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine
added a commit
that referenced
this issue
Aug 17, 2021
…ved object references (Part 2) (#108291) (#108991) ## Summary This is part 2 to addressing the issue seen here: #101975 (Part 1 #107064) This adds the alerting migration scripts and unit tests for exception list containers on Kibana startup for `7.15.0` We only migrate if we find these conditions and cases: - `exceptionLists` are an `array` and not `null`, `undefined`, or malformed data. - The exceptionList item is an `object` and its `id` is a `string` and not `null`, `undefined`, or malformed data - The existing references do not already have an exceptionItem reference already found within it. We migrate on the common use case - The saved object references do not exist but we have exceptionList items with the id's to create the saved object references, 👍 so we migrate. - The alert contains no exception list items, in which case we have nothing to migrate We do these additional (mis-use) cases and steps as well. These should _NOT_ be common things that happen but we safe guard for them here: - If the migration is run twice we are idempotent and do _NOT_ add duplicates list items or remove items. - If the migration was partially successful but re-run a second time, we only add what is missing. Again no duplicates or removed items should occur. - If the `exceptionLists` contains invalid data shape or not enough information to migrate, we filter it out and ignore it - If the saved object references already exists and contains a different or foreign value, we will retain the foreign reference(s) and still migrate. ## Manual testing There are unit tests but for any manual testing or verification you can do the following: Create a few alerts through the `security_solution` application with exception lists <img width="1775" alt="Screen Shot 2021-08-11 at 5 42 31 PM" src="https://user-images.githubusercontent.com/1151048/129117377-61b17fcf-ad01-4405-bbfe-42d97a6f7654.png"> Use the dev tools to de-migrate as well as to test end to end like so: ```json # First get an "_id" with an exceptions list like below. Mine I found was: "alert:38482620-ef1b-11eb-ad71-7de7959be71c": GET .kibana/_search { "query": { "terms": { "alert.alertTypeId": [ "siem.signals" ] } }, "size": 10000 } ``` With Kibana running downgrade and remove the references as a test: ```json # Set saved object array references as empty arrays and set our migration version to be 7.14.0 POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; ctx._source.references = [] """, "lang": "painless" } } # Double check the references is empty and the version is 7.14.0 GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` Reload the alert in the `security_solution` and notice you get these errors until you restart Kibana to cause a migration moving forward ```sh server log [17:35:16.914] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"} server log [17:35:16.914] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[] server log [17:35:16.915] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"} server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"} server log [17:35:16.940] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[] server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"} ``` Restart Kibana and you should no longer have errors in the Kibana console. If you do this query in dev tools ```json # Check that the `migrationVersion` is `7.15.0` and that we have a `references` array filled out with the correct structure GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` You should notice that you now have a `references` array filled out: ```json "references" : [ { "name" : "param:exceptionsList_0", "id" : "endpoint_list", "type" : "exception-list-agnostic" }, { "name" : "param:exceptionsList_1", "id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c", "type" : "exception-list" } ], "migrationVersion" : { "alert" : "7.15.0" } ``` For testing [idempotentence](https://en.wikipedia.org/wiki/Idempotence) Run just this to downgrade and restart Kibana and you should notice on a GET that we do not have anything extra in the references array: ```json # Set our migration version to be 7.14.0 only POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; """, "lang": "painless" } } # Double check the `references` is still there, and we do not get errors or changes to `references` after we restart Kibana GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` For testing foreign keys: ```json # Set saved object array references to foreign keys and set our migration version to be 7.14.0 POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.14.0"; ctx._source.references = [["name" : "foreign", "id" : "123", "type" : "some-type"]]; """, "lang": "painless" } } ``` Restart, ensure no errors in Kibana console and do a get call to ensure we have the foreign mixed with valid values: ```json GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c ``` Should return this data: ```json "type" : "alert", "references" : [ { "name" : "foreign", "id" : "123", "type" : "some-type" }, { "name" : "param:exceptionsList_0", "id" : "endpoint_list", "type" : "exception-list-agnostic" }, { "name" : "param:exceptionsList_1", "id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c", "type" : "exception-list" } ] ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: Frank Hassanabad <frank.hassanabad@elastic.co>
|
@peluja1012 @MadameSheema Is this one on the list for QA to test? |
|
@yctercero can we please sync about this when you are available? Thanks :) |
|
Work was complete. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Once usage of the
SO Reference Arrayis made available to alerts (#87992), we can then start leveraging this field for joiningException Lists, which will allow for faster lookups (#85173) and the ability for theSO Mgmt Import/ExportUI to export Exception Lists along with Rules.This effort can be bundled with the SO Migrations necessary for moving from the single
siem.signalsRule Type to separate rule types for each the Security Detection Rule Types (Query, ML, EQL, etc...).The text was updated successfully, but these errors were encountered: