Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] [Feat] Add Bulk Events to Timeline. #142737

Merged
merged 20 commits into from
Nov 9, 2022

Conversation

logeekal
Copy link
Contributor

@logeekal logeekal commented Oct 5, 2022

Summary

This PR implements functionality to add multiple events to the timeline. It is implements :

  1. Selected events ( max 4000 ) can be added to the timeline.

Implementation.

  1. At present, it only support adding multiple records in the timeline as kql Filters. [Security Solution] Add is one of operator for usage with DataProvider's QueryMatch value. #142436 is in progress to implement is-one-of operator in the data provider. Once that is moved to main, we can change value of prefer parameter to send the IDs in dataProvider rather than filter.

If you would like to test it with #142436, please clone : https://github.com/logeekal/kibana/tree/bulk_actions_add_timeline_with_is_one_of

  1. Below is the demonstration how it works.
Screen.Recording.2022-10-31.at.15.51.56.mov

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@logeekal logeekal force-pushed the feat/bulk_actions_add_timeline branch from 814f338 to 9468db9 Compare October 6, 2022 13:12
@logeekal logeekal marked this pull request as ready for review October 6, 2022 13:33
@logeekal logeekal requested a review from a team October 6, 2022 13:33
@logeekal logeekal requested review from a team as code owners October 6, 2022 13:33
@logeekal logeekal self-assigned this Oct 6, 2022
@logeekal logeekal added v8.6.0 release_note:feature Makes this part of the condensed release notes labels Oct 6, 2022
@logeekal logeekal changed the title Feat/bulk actions add timeline [Security Solution] [Feat] Add Bulk Events to Timeline. Oct 6, 2022
@logeekal logeekal force-pushed the feat/bulk_actions_add_timeline branch from d9da37d to 2a59c5e Compare October 17, 2022 09:38
@logeekal logeekal requested review from a team as code owners October 17, 2022 09:38
};

const addToCaseBulkActions = useBulkAddToCaseActions();
const bulkActions = useMemo(
Copy link
Contributor Author

@logeekal logeekal Oct 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bulk Actions prop has been moved up the tree so that it is easier for implementing function to add their own bulk actions.

@logeekal

This comment was marked as outdated.

@logeekal logeekal added backport:skip This commit does not require backporting ci:cloud-redeploy Always create a new Cloud deployment ci:cloud-deploy Create or update a Cloud deployment and removed ci:cloud-redeploy Always create a new Cloud deployment labels Oct 17, 2022
Copy link
Contributor

@dplumlee dplumlee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good for detections alerts area code

@logeekal logeekal force-pushed the feat/bulk_actions_add_timeline branch from 5830cbf to 7c36642 Compare October 19, 2022 08:56
@@ -108,6 +108,7 @@ const SessionsViewComponent: React.FC<SessionsComponentsProps> = ({
pageFilters={sessionsFilter}
defaultModel={getSessionsDefaultModel(columns, defaultColumns)}
end={endDate}
bulkActions={false}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @logeekal,

I'm just curious, is there any particular reason to not add that feature on the Sessions Tab as well? It seems the only place to be left out.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @opauloh , you were right. I had missed it. But I have now added it to session viewer as well. Thanks for pointing it out.

@logeekal logeekal requested a review from a team as a code owner October 28, 2022 09:10
@logeekal logeekal requested a review from a team October 28, 2022 09:10
@logeekal logeekal requested review from a team as code owners October 28, 2022 09:10
@logeekal logeekal force-pushed the feat/bulk_actions_add_timeline branch from 293fb93 to 89047d2 Compare November 8, 2022 11:55
@logeekal
Copy link
Contributor Author

logeekal commented Nov 8, 2022

Files by Code Owner

elastic/awp-viz

  • x-pack/plugins/security_solution/public/common/components/sessions_viewer/index.tsx

elastic/security-detections-response

  • x-pack/plugins/security_solution/cypress/objects/filter.ts
  • x-pack/plugins/security_solution/cypress/screens/common/controls.ts

elastic/security-detections-response-alerts

  • x-pack/plugins/security_solution/public/detections/components/rules/rule_preview/preview_histogram.tsx

elastic/security-detections-response-rules

  • x-pack/plugins/security_solution/public/detections/components/rules/rule_preview/preview_histogram.tsx

elastic/security-engineering-productivity

  • x-pack/test/security_solution_cypress/es_archives/bulk_process/data.json
  • x-pack/test/security_solution_cypress/es_archives/bulk_process/mappings.json

elastic/security-solution

  • x-pack/plugins/security_solution/common/constants.ts
  • x-pack/plugins/security_solution/cypress/e2e/detection_alerts/investigate_in_timeline.cy.ts
  • x-pack/plugins/security_solution/cypress/e2e/hosts/events_viewer.cy.ts
  • x-pack/plugins/security_solution/cypress/e2e/hosts/session_viewer.cy.ts
  • x-pack/plugins/security_solution/cypress/objects/filter.ts
  • x-pack/plugins/security_solution/cypress/screens/common/controls.ts
  • x-pack/plugins/security_solution/cypress/screens/hosts/events.ts
  • x-pack/plugins/security_solution/cypress/screens/hosts/main.ts
  • x-pack/plugins/security_solution/cypress/tasks/common/event_table.ts
  • x-pack/plugins/security_solution/cypress/tasks/hosts/main.ts
  • x-pack/plugins/security_solution/cypress/tasks/search_bar.ts
  • x-pack/plugins/security_solution/public/common/components/events_tab/events_query_tab_body.test.tsx
  • x-pack/plugins/security_solution/public/common/components/events_tab/events_query_tab_body.tsx
  • x-pack/plugins/security_solution/public/common/components/events_viewer/index.test.tsx
  • x-pack/plugins/security_solution/public/common/components/events_viewer/index.tsx
  • x-pack/plugins/security_solution/public/common/components/sessions_viewer/index.tsx
  • x-pack/plugins/security_solution/public/common/demo_data/timeline.ts
  • x-pack/plugins/security_solution/public/common/mock/global_state.ts
  • x-pack/plugins/security_solution/public/common/mock/timeline_results.ts
  • x-pack/plugins/security_solution/public/common/store/data_table/actions.ts
  • x-pack/plugins/security_solution/public/common/store/data_table/defaults.ts
  • x-pack/plugins/security_solution/public/common/store/data_table/epic_local_storage.test.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.test.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/index.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/timeline_actions/investigate_in_timeline_action.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/timeline_actions/use_add_bulk_to_timeline.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/translations.ts
  • x-pack/plugins/security_solution/public/detections/components/rules/rule_preview/preview_histogram.tsx
  • x-pack/plugins/security_solution/public/detections/pages/detection_engine/detection_engine.test.tsx
  • x-pack/plugins/security_solution/public/timelines/containers/index.tsx
  • x-pack/plugins/security_solution/public/timelines/containers/local_storage/index.test.ts
  • x-pack/plugins/security_solution/public/timelines/containers/local_storage/index.tsx
  • x-pack/plugins/security_solution/public/timelines/store/timeline/actions.ts

elastic/security-threat-hunting

  • x-pack/plugins/security_solution/cypress/objects/filter.ts
  • x-pack/plugins/security_solution/cypress/screens/common/controls.ts

elastic/security-threat-hunting-explore

  • x-pack/plugins/security_solution/cypress/e2e/hosts/events_viewer.cy.ts
  • x-pack/plugins/security_solution/cypress/e2e/hosts/session_viewer.cy.ts
  • x-pack/plugins/security_solution/cypress/screens/hosts/events.ts
  • x-pack/plugins/security_solution/cypress/screens/hosts/main.ts
  • x-pack/plugins/security_solution/cypress/tasks/hosts/main.ts

elastic/security-threat-hunting-investigations

  • x-pack/plugins/security_solution/cypress/e2e/detection_alerts/investigate_in_timeline.cy.ts
  • x-pack/plugins/security_solution/public/common/components/events_viewer/index.test.tsx
  • x-pack/plugins/security_solution/public/common/components/events_viewer/index.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.test.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/actions.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/index.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/timeline_actions/investigate_in_timeline_action.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/timeline_actions/use_add_bulk_to_timeline.tsx
  • x-pack/plugins/security_solution/public/detections/components/alerts_table/translations.ts
  • x-pack/plugins/security_solution/public/timelines/containers/index.tsx
  • x-pack/plugins/security_solution/public/timelines/containers/local_storage/index.test.ts
  • x-pack/plugins/security_solution/public/timelines/containers/local_storage/index.tsx
  • x-pack/plugins/security_solution/public/timelines/store/timeline/actions.ts
  • x-pack/plugins/timelines/public/components/t_grid/integrated/index.tsx
  • x-pack/plugins/timelines/public/container/index.tsx
  • x-pack/plugins/timelines/public/index.ts
  • x-pack/plugins/timelines/public/mock/global_state.ts
  • x-pack/plugins/timelines/public/mock/mock_timeline_data.ts
  • x-pack/plugins/timelines/public/mock/t_grid.tsx
  • x-pack/plugins/timelines/public/store/t_grid/actions.ts
  • x-pack/plugins/timelines/public/store/t_grid/defaults.ts
  • x-pack/plugins/timelines/public/store/t_grid/model.ts
  • x-pack/plugins/timelines/public/store/t_grid/reducer.ts

Copy link
Contributor

@angorayc angorayc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally, all looks good, thank you @logeekal


it('Adding multiple alerts to the timeline should be successful', () => {
// select all visible events
investigateFirstPageEventsInTimeline();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertions on tests should not be wrapped inside the tasks methods, should be explicitly written in the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema .. Sure, I have now removed the assertions from tasks. Please check new updates.

});

it('When selected all alerts are selected should be successfull', () => {
investigateAllEventsInTimeline();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertions on tests should not be wrapped inside the tasks methods, should be explicitly written in the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema .. Sure, I have now removed the assertions from tasks. Please check new updates.


it('Adding multiple events to the timeline should be successful', () => {
// select all visible events
investigateFirstPageEventsInTimeline();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertions on tests should not be wrapped inside the tasks methods, should be explicitly written in the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema .. Sure, I have now removed the assertions from tasks. Please check new updates.

});

it('When selected all events are selected, bulk action should be disabled', () => {
investigateAllEventsInTimeline();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertions on tests should not be wrapped inside the tasks methods, should be explicitly written in the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema .. Sure, I have now removed the assertions from tasks. Please check new updates.


it('Adding multiple events to the timeline should be successful', () => {
// select all visible events
investigateFirstPageEventsInTimeline();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertions on tests should not be wrapped inside the tasks methods, should be explicitly written in the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

});

it('When selected all events are selected, bulk action should be disabled', () => {
investigateAllEventsInTimeline();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertions on tests should not be wrapped inside the tasks methods, should be explicitly written in the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@@ -62,9 +68,39 @@ describe('Alerts timeline', () => {
});

it('Add an empty property to default timeline', () => {
// add condition to make sure the field is empty
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are lot of records and we want to make sure that first row has empty value for file.name . Safest method to ensure that is to add a filter.

Now this situation arises sometimes when this test runs after another test which hasn't unloaded their own data. It looks like cleankibana does not clear the data uploaded by esarchiver

cy.get(ADD_FILTER_FORM_OPERATOR_FIELD).click();
cy.get(ADD_FILTER_FORM_OPERATOR_OPTION_IS).click();
cy.get(ADD_FILTER_FORM_FILTER_VALUE_INPUT).type(value);
if (!operator) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are trying to avoid conditionals inside tasks methods.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand avoiding conditional inside assertions as they may become in-deterministic but is there particular reason to avoid conditionals in tasks.

Nonetheless, this condition can still be removed by making some changes but the next condition if (value) cannot be removed as filling the value in the filter form depends on the operator you have selected. Please see below screenshots for comparison.

Value Field visible Value Field not visible
image image

cleanKibana();
login();
createCustomRuleEnabled(getNewRule());
esArchiverLoad('bulk_process');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the file we are using is huge, executing the loading and unloading of it is around one minute. As we are doing this in several tests, we are increasing the test execution time a lot. If it is not possible to reduce it, we should consider putting all the tests that are using it in the same cypress file to load it and unload it just once reducing the time execution.

Copy link
Contributor Author

@logeekal logeekal Nov 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema , it makes sense.. I have consolidated all tests in a single test suite. Please check new updates. Thanks.

Here is new test file: https://github.com/elastic/kibana/pull/142737/files#diff-126a474680b59e71fa3835f6f722e95dc39ff263e3b7b32fae883ea5ba3c0dd5

});

beforeEach(() => {
visit(ALERTS_URL);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better in terms of execution time, to close the timeline each time instead of reloading the page and waiting for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema You are right. Updated.

@logeekal
Copy link
Contributor Author

logeekal commented Nov 9, 2022

Thanks @MadameSheema for detailed feedback.

Copy link
Member

@MadameSheema MadameSheema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@kibana-ci
Copy link
Collaborator

kibana-ci commented Nov 9, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 3204 3205 +1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
timelines 349 350 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 9.6MB 9.6MB +16.2KB
timelines 74.1KB 74.7KB +550.0B
total +16.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 50.6KB 50.7KB +120.0B
timelines 138.5KB 138.8KB +292.0B
total +412.0B
Unknown metric groups

API count

id before after diff
timelines 458 462 +4

ESLint disabled in files

id before after diff
osquery 1 2 +1

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
fleet 59 65 +6
osquery 108 113 +5
securitySolution 440 446 +6
total +19

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
fleet 67 73 +6
osquery 109 115 +6
securitySolution 517 523 +6
total +20

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @logeekal

@logeekal logeekal enabled auto-merge (squash) November 9, 2022 14:16
@logeekal logeekal merged commit 92d907c into elastic:main Nov 9, 2022
jloleysens added a commit to jloleysens/kibana that referenced this pull request Nov 9, 2022
* main:
  [Lens] Rearrange options (elastic#144891)
  [Actionable Observability] Integrate alert search bar on rule details page (elastic#144718)
  [Security Solution] [Exceptions] Adds options to create a shared exception list and to create a single item from the manage exceptions view (elastic#144575)
  [Actionable Observability] Add context.alertDetailsUrl to connector template for Uptime > Monitor status & Uptime TLS rules (elastic#144740)
  [Security Solution] [Feat] Add Bulk Events to Timeline. (elastic#142737)
  [TIP] Env specific cypress config (elastic#144894)
  skip flaky suite (elastic#144885)
  [Enterprise Search] Fixes Search Index page to go blank when connection lost (elastic#144022)
  [Cloud Posture] track findings pages (elastic#144822)
  [ContentManagement] Inspector flyout (elastic#144240)
  [Cloud Posture] Dashboard Redesign - data counter cards (elastic#144565)
  [TIP] Run e2e pipeline on CI (elastic#144776)
  [Guided onboarding] Config updates for the Security guide (elastic#144844)
  Cleanup unused code for claiming tasks by id (elastic#144408)
  Ping the response-ops team whenever a new connector type is registered (elastic#144736)
@KOTungseth KOTungseth added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Nov 17, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting ci:cloud-deploy Create or update a Cloud deployment release_note:feature Makes this part of the condensed release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.6.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants