Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] [Trigger Actions] Alert Table Refactoring #149128

Merged
merged 92 commits into from Feb 22, 2023

Conversation

logeekal
Copy link
Contributor

@logeekal logeekal commented Jan 18, 2023

Summary

This PR replaces the existing Alert Table used in Security solution & cases with that of triggers-actions-ui.

Ideally, this PR does make any changes to the functionality of the product and from user perspective. Nothing should Change.

‼️ Note for @elastic/security-threat-hunting-explore : This PR makes no changes to the table used in Host/Users page.

Things to test and changes

  • @elastic/actionable-observability

    • The changes in observability plugin are done to accommodate the changes in the API of triggers-actions-ui alert table.
    • Requesting you to do desk-testing this PR once by using Alert Table as you do on the daily basis.
  • @elastic/response-ops

    • changes have been done in API as security-solution needed some parameters/values to achieve the parity in the functionality as compared to the current alert Table.

@logeekal logeekal requested a review from a team as a code owner February 21, 2023 09:11
Copy link
Member

@MadameSheema MadameSheema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of thanks for addressing all the suggestions :)

Copy link
Contributor

@semd semd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally, and everything works as expected.
Thanks @logeekal for taking the time to address CellActions changes.
LGTM! 🚀

@cnasikas
Copy link
Member

@logeekal The alerts table test was skipped in main (#151688). Feel free to ignore it for now. We will fix it soon.

@logeekal
Copy link
Contributor Author

Thanks @cnasikas, I did notice it and thought that this PR is old and I was checking our code might have fixed it. But I will skip it for now.. Thanks.

@logeekal logeekal enabled auto-merge (squash) February 22, 2023 15:23
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 3682 3702 +20

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
triggersActionsUi 531 503 -28

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
triggersActionsUi 11 10 -1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
cases 367.1KB 367.1KB +38.0B
observability 596.0KB 596.0KB +10.0B
securitySolution 13.7MB 13.8MB +45.3KB
triggersActionsUi 919.6KB 922.5KB +2.9KB
total +48.2KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 53.0KB 53.3KB +307.0B
Unknown metric groups

API count

id before after diff
triggersActionsUi 560 532 -28

ESLint disabled in files

id before after diff
securitySolution 76 77 +1

ESLint disabled line counts

id before after diff
triggersActionsUi 123 124 +1

Total ESLint disabled count

id before after diff
securitySolution 503 504 +1
triggersActionsUi 129 130 +1
total +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@logeekal logeekal merged commit fe04a4c into elastic:main Feb 22, 2023
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Feb 22, 2023
@stephmilovic
Copy link
Contributor

I think this may have broken building block highlighting?

wtf.mov

@logeekal
Copy link
Contributor Author

logeekal commented Feb 28, 2023

I think this may have broken building block highlighting?

@stephmilovic , created bug for the same : [Security Solution] New Alert table may have broken building block highlighting.

michaelolo24 pushed a commit that referenced this pull request May 8, 2023
… from timestamp instead of @timestamp field (#156884)

## Summary

In Kibana 8.8 we've done a huge refactor of the alert table (see
[PR](#149128)).
The new table's trigger actions are no longer some of the Security
Solution server side methods that were adding a `timestamp` field to the
response (see
[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/format_timeline_data.ts#L28)).
That `timestamp` field was basically a duplication of the `@timestamp`
field (done via [this helper
function](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/get_timestamp.ts#L12))

Running the stack locally, you would see that clicking on the
`Investigate in timeline` icon button or ANY alerts in the alerts table
(pretty new or months/years old) would bring the timeline flyout with
always the `to` value being the `current date`, and the `from` value
being the `current date - kibana.alert.rule.from`.
The `timetamp` field
[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsx#L158)
does not exists, so we always fall back to `new Date()` (unless you're
looking at an alert generated by a threshold rule, a new terms rule or a
suppressed alert).

This PR fixes the issue by retrieving the `@timestamp` field instead of
the unwanted `timestamp` field.

More work will be done in the future to actually entirely remove and
cleanup the `timestamp` field (see [this WIP
ticket](#156879)).

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

Before (recorded on May 4th and we're looking at an alert generated on
May 3rd)


https://user-images.githubusercontent.com/17276605/236509848-a5b0a363-c9c5-4d80-a139-84e3df3a1bd6.mov

After


https://user-images.githubusercontent.com/17276605/236509884-74805cef-ccf2-4b09-a174-2fcb6b75d4bb.mov

Closes #126077
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request May 8, 2023
… from timestamp instead of @timestamp field (elastic#156884)

## Summary

In Kibana 8.8 we've done a huge refactor of the alert table (see
[PR](elastic#149128)).
The new table's trigger actions are no longer some of the Security
Solution server side methods that were adding a `timestamp` field to the
response (see
[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/format_timeline_data.ts#L28)).
That `timestamp` field was basically a duplication of the `@timestamp`
field (done via [this helper
function](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/get_timestamp.ts#L12))

Running the stack locally, you would see that clicking on the
`Investigate in timeline` icon button or ANY alerts in the alerts table
(pretty new or months/years old) would bring the timeline flyout with
always the `to` value being the `current date`, and the `from` value
being the `current date - kibana.alert.rule.from`.
The `timetamp` field
[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsx#L158)
does not exists, so we always fall back to `new Date()` (unless you're
looking at an alert generated by a threshold rule, a new terms rule or a
suppressed alert).

This PR fixes the issue by retrieving the `@timestamp` field instead of
the unwanted `timestamp` field.

More work will be done in the future to actually entirely remove and
cleanup the `timestamp` field (see [this WIP
ticket](elastic#156879)).

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

Before (recorded on May 4th and we're looking at an alert generated on
May 3rd)

https://user-images.githubusercontent.com/17276605/236509848-a5b0a363-c9c5-4d80-a139-84e3df3a1bd6.mov

After

https://user-images.githubusercontent.com/17276605/236509884-74805cef-ccf2-4b09-a174-2fcb6b75d4bb.mov

Closes elastic#126077

(cherry picked from commit 114c98d)
kibanamachine added a commit that referenced this pull request May 9, 2023
…pulled from timestamp instead of @timestamp field (#156884) (#157114)

# Backport

This will backport the following commits from `main` to `8.8`:
- [[Security Solution] fix from-to values investigate in timeline pulled
from timestamp instead of @timestamp field
(#156884)](#156884)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Philippe
Oberti","email":"philippe.oberti@elastic.co"},"sourceCommit":{"committedDate":"2023-05-08T23:44:14Z","message":"[Security
Solution] fix from-to values investigate in timeline pulled from
timestamp instead of @timestamp field (#156884)\n\n## Summary\r\n\r\nIn
Kibana 8.8 we've done a huge refactor of the alert table
(see\r\n[PR](#149128 new
table's trigger actions are no longer some of the Security\r\nSolution
server side methods that were adding a `timestamp` field to
the\r\nresponse
(see\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/format_timeline_data.ts#L28)).\r\nThat
`timestamp` field was basically a duplication of the
`@timestamp`\r\nfield (done via [this
helper\r\nfunction](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/get_timestamp.ts#L12))\r\n\r\nRunning
the stack locally, you would see that clicking on the\r\n`Investigate in
timeline` icon button or ANY alerts in the alerts table\r\n(pretty new
or months/years old) would bring the timeline flyout with\r\nalways the
`to` value being the `current date`, and the `from` value\r\nbeing the
`current date - kibana.alert.rule.from`.\r\nThe `timetamp`
field\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsx#L158)\r\ndoes
not exists, so we always fall back to `new Date()` (unless
you're\r\nlooking at an alert generated by a threshold rule, a new terms
rule or a\r\nsuppressed alert).\r\n\r\nThis PR fixes the issue by
retrieving the `@timestamp` field instead of\r\nthe unwanted `timestamp`
field.\r\n\r\nMore work will be done in the future to actually entirely
remove and\r\ncleanup the `timestamp` field (see [this
WIP\r\nticket](#156879
Checklist\r\n\r\n- [ ] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\nBefore
(recorded on May 4th and we're looking at an alert generated on\r\nMay
3rd)\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17276605/236509848-a5b0a363-c9c5-4d80-a139-84e3df3a1bd6.mov\r\n\r\nAfter\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17276605/236509884-74805cef-ccf2-4b09-a174-2fcb6b75d4bb.mov\r\n\r\nCloses
#126077
Hunting","Team: SecuritySolution","Team:Threat
Hunting:Investigations","v8.8.0","v8.9.0"],"number":156884,"url":"#156884
Solution] fix from-to values investigate in timeline pulled from
timestamp instead of @timestamp field (#156884)\n\n## Summary\r\n\r\nIn
Kibana 8.8 we've done a huge refactor of the alert table
(see\r\n[PR](#149128 new
table's trigger actions are no longer some of the Security\r\nSolution
server side methods that were adding a `timestamp` field to
the\r\nresponse
(see\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/format_timeline_data.ts#L28)).\r\nThat
`timestamp` field was basically a duplication of the
`@timestamp`\r\nfield (done via [this
helper\r\nfunction](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/get_timestamp.ts#L12))\r\n\r\nRunning
the stack locally, you would see that clicking on the\r\n`Investigate in
timeline` icon button or ANY alerts in the alerts table\r\n(pretty new
or months/years old) would bring the timeline flyout with\r\nalways the
`to` value being the `current date`, and the `from` value\r\nbeing the
`current date - kibana.alert.rule.from`.\r\nThe `timetamp`
field\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsx#L158)\r\ndoes
not exists, so we always fall back to `new Date()` (unless
you're\r\nlooking at an alert generated by a threshold rule, a new terms
rule or a\r\nsuppressed alert).\r\n\r\nThis PR fixes the issue by
retrieving the `@timestamp` field instead of\r\nthe unwanted `timestamp`
field.\r\n\r\nMore work will be done in the future to actually entirely
remove and\r\ncleanup the `timestamp` field (see [this
WIP\r\nticket](#156879
Checklist\r\n\r\n- [ ] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\nBefore
(recorded on May 4th and we're looking at an alert generated on\r\nMay
3rd)\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17276605/236509848-a5b0a363-c9c5-4d80-a139-84e3df3a1bd6.mov\r\n\r\nAfter\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17276605/236509884-74805cef-ccf2-4b09-a174-2fcb6b75d4bb.mov\r\n\r\nCloses
#126077
Solution] fix from-to values investigate in timeline pulled from
timestamp instead of @timestamp field (#156884)\n\n## Summary\r\n\r\nIn
Kibana 8.8 we've done a huge refactor of the alert table
(see\r\n[PR](#149128 new
table's trigger actions are no longer some of the Security\r\nSolution
server side methods that were adding a `timestamp` field to
the\r\nresponse
(see\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/format_timeline_data.ts#L28)).\r\nThat
`timestamp` field was basically a duplication of the
`@timestamp`\r\nfield (done via [this
helper\r\nfunction](https://github.com/elastic/kibana/blob/main/x-pack/plugins/timelines/server/search_strategy/timeline/factory/helpers/get_timestamp.ts#L12))\r\n\r\nRunning
the stack locally, you would see that clicking on the\r\n`Investigate in
timeline` icon button or ANY alerts in the alerts table\r\n(pretty new
or months/years old) would bring the timeline flyout with\r\nalways the
`to` value being the `current date`, and the `from` value\r\nbeing the
`current date - kibana.alert.rule.from`.\r\nThe `timetamp`
field\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsx#L158)\r\ndoes
not exists, so we always fall back to `new Date()` (unless
you're\r\nlooking at an alert generated by a threshold rule, a new terms
rule or a\r\nsuppressed alert).\r\n\r\nThis PR fixes the issue by
retrieving the `@timestamp` field instead of\r\nthe unwanted `timestamp`
field.\r\n\r\nMore work will be done in the future to actually entirely
remove and\r\ncleanup the `timestamp` field (see [this
WIP\r\nticket](#156879
Checklist\r\n\r\n- [ ] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\nBefore
(recorded on May 4th and we're looking at an alert generated on\r\nMay
3rd)\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17276605/236509848-a5b0a363-c9c5-4d80-a139-84e3df3a1bd6.mov\r\n\r\nAfter\r\n\r\n\r\nhttps://user-images.githubusercontent.com/17276605/236509884-74805cef-ccf2-4b09-a174-2fcb6b75d4bb.mov\r\n\r\nCloses
#126077"}}]}]
BACKPORT-->

Co-authored-by: Philippe Oberti <philippe.oberti@elastic.co>
@vitaliidm
Copy link
Contributor

vitaliidm commented Jun 2, 2023

Hey @logeekal, it looks like we found related issue: #158965
Is this something that could be still configured on Security Solution side, or need to be done in Trigger Actions?

@kqualters-elastic
Copy link
Contributor

kqualters-elastic commented Jun 8, 2023

Another related issue: #159276 I have no idea what oldAlertsData or ecsAlertsData even are, nor what the difference is or how long they need to be supported and what if any the migration plan is, when to use one vs the other, kinda goes beyond a (non-existent) tech debt ticket as mentioned here: #149128 (comment) . Probably should have released this behind a feature flag defaulted to true or something

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Team:Threat Hunting:Investigations Security Solution Investigations Team v8.8.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet