Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM] Use ECS categorisation for Authentication widgets #60734

Merged
merged 8 commits into from
Mar 23, 2020

Conversation

tsg
Copy link
Contributor

@tsg tsg commented Mar 20, 2020

Summary

The current authentication widgets make use of event.type which currently has two possible values: authentication_success and authentication_failure. In ECS 1.4, event.type is being changed to contain start/stop information.

The good news is, in both ECS < 1.4 and >=1.4, event.category is set to authentication and event.outcome is set to success / failure.

After discussing with @MikePaquette, we have decided to ignore event.type in our queries and only rely on event.category and event.outcome. This means that we can have the same queries work on both old and new data.

There are two drawbacks:

  • If the date source creates multiple events per authentication, separated by event.type, this can result in counting more authentications. To our knowledge, that's not currently the case with our data sources.
  • In the current version, the Endgame integration (Phase 0) doesn't fill in event.outcome which means that we don't display those events. This should be fixed on that integration.

Once we get to 8.0, we should add an extra condition for event.type.

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@tsg tsg added the Team:SIEM label Mar 20, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@tsg
Copy link
Contributor Author

tsg commented Mar 20, 2020

@elasticmachine merge upstream

@tsg tsg added release_note:skip Skip the PR/issue when compiling release notes v7.7.0 labels Mar 20, 2020
@FrankHassanabad
Copy link
Contributor

@elasticmachine merge upstream

@elasticmachine elasticmachine requested a review from a team as a code owner March 20, 2020 19:53
@FrankHassanabad
Copy link
Contributor

FrankHassanabad commented Mar 20, 2020

To run this one unit test failure:

cd kibana/x-pack
node scripts/jest.js x-pack/legacy/plugins/siem/server/lib/kpi_hosts/query_authentication.dsl.test.ts

Side note: if you want to run all jest tests:

cd kibana/x-pack
node scripts/jest.js siem

Diff/PR of the fix for the test if you want it:
tsg#1

Copy link
Contributor

@FrankHassanabad FrankHassanabad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping us out during the release cycle with pull requests against Kibana. I put a few notes on how to update your tests and I ran this locally and tested it and it looks like it is working well.

LGTM 👍

@tsg tsg requested a review from angorayc March 21, 2020 13:13
@tsg
Copy link
Contributor Author

tsg commented Mar 21, 2020

@elasticmachine merge upstream

@tsg tsg merged commit 7bafeb1 into elastic:master Mar 23, 2020
tsg added a commit to tsg/kibana that referenced this pull request Mar 23, 2020
* Update the Authentication histogram to use categorization fields

* linting

* Use categorization fields for the Authentications table

* Use event.outcome for authentications KPIs

* Adjust mock to fix unit test

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Mar 23, 2020
* master: (39 commits)
  [APM]Create custom link from Trace summary (elastic#59648)
  [ML] Fixing app clean up (elastic#60853)
  [SIEM] Use ECS categorisation for Authentication widgets (elastic#60734)
  [NP] Remove kbnUrl usage in discover/dashboard/visualize (elastic#60016)
  Skip failing test
  [Uptime]Update fetch effect failed action handling (elastic#60742)
  [npm] upgrade elastic/maki (elastic#60829)
  [Uptime] Add Settings Page (elastic#53550)
  [APM] service maps: avoid unnecesary `useDeepObjectIdentity` (elastic#60836)
  [Index management] Re-enable index template tests (elastic#60780)
  Fixed UI/UX issues: alerts delete confirmation, combobox behaviors (elastic#60703)
  [SIEM] Fix patching of ML Rules (elastic#60830)
  [APM] Service Map - Separate overlapping edges by rotating nodes (elastic#60477)
  [Alerting] fix flaky test for index threshold grouping (elastic#60792)
  [SIEM][Detection Engine] Adds test scripts for machine learning feature
  Flatten child api response for resolver (elastic#60810)
  Change "url" to "urls" in APM agent instructions (elastic#60790)
  [DOCS] Updates API requests and examples (elastic#60695)
  [SIEM] [Cases] Create case from timeline (elastic#60711)
  [Lens] Resetting a layer generates new suggestions (elastic#60674)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Mar 23, 2020
…s/kibana into alerting/fix-duration-flaky-test

* 'alerting/fix-duration-flaky-test' of github.com:gmmorris/kibana:
  [ML] Fixing app clean up (elastic#60853)
  [SIEM] Use ECS categorisation for Authentication widgets (elastic#60734)
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

tsg added a commit that referenced this pull request Mar 24, 2020
…0890)

* Update the Authentication histogram to use categorization fields

* linting

* Use categorization fields for the Authentications table

* Use event.outcome for authentications KPIs

* Adjust mock to fix unit test

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:SIEM v7.7.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants