[SIEMDPOINT] Move endpoint to siem #66907

XavierM · 2020-05-18T16:10:22Z

Summary

Move endpoint to siem

Add in kibana dev config

xpack.ingestManager.enabled: true
xpack.ingestManager.epm.enabled: true
xpack.ingestManager.fleet.enabled: true

New routes:

…o-siem

elasticmachine · 2020-05-18T16:11:19Z

Pinging @elastic/endpoint-management (Team:Endpoint Management)

elasticmachine · 2020-05-18T16:11:20Z

Pinging @elastic/endpoint-data-visibility-team (Team:Endpoint Data Visibility)

elasticmachine · 2020-05-18T16:11:22Z

Pinging @elastic/siem (Team:SIEM)

elasticmachine · 2020-05-18T16:11:23Z

Pinging @elastic/endpoint-response (Team:Endpoint Response)

oatkiller · 2020-05-18T17:53:07Z

Action Items / Debrief

Here's some stuff from notes I took while working w/ @XavierM on this:

Action items:

Re-enable and fix tests

We kept as many tests enabled as we could. Unfortunately we did disable some. We should try and re-enable tests as soon as possible. In some cases, we may need to integrate a feature more before testing.

Consolidate ‘StartDeps’.

The dependencies that Endpoint needed from NP aren’t exactly the same as the ones SIEM needed. SIEM and Endpoint each had a type for these deps. We added the values that Endpoint needed to the SIEM type (and implementation.) But many places still refer to the Endpoint type. Should we have 1 type plugin-wide? Should each ‘supplugin’ define its own type? TBD

UI ‘links’ are implemented in a different way in SIEM vs Endpoint.

On the Endpoint side @paul-tavares implemented a way to link to places in Kibana / other routes. Let’s compare that w/ what SIEM has and consider consolidating approaches.

Consider formalizing (and finishing) the 'Subplugin' idea

@XavierM proposed the idea of ‘subplugins’ which divide functionality within a plugin. This was semi-implemented in the ‘public’ code, but its not there for the server side. We should define this pattern, and if we like it, argue for adopting it plugin-wide.

Come to agreement on eslint rules that aren’t part of the Kibana standard.

When development started on the endpoint plugin, we were advised not to deviate from the Kibana standard lint rules. The idea is that code that’s valid anywhere in Kibana should be valid everywhere. This should allow us to, for example, copy Endpoint code into SIEM w/o changing it. The SIEM team has some custom lint rules. When the Endpoint code was moved into SIEM, Xavier changed the code to meet SIEM eslint rules. Let’s discuss as a team (and perhaps w/ the Kibana arch group) these lint rules. If we deviate from Kibana in a way that affects all code in the plugin, then we should agree on those rules as a team.

Explain certain Endpoint code concepts to the SIEM team

The Endpoint code has some novel patterns. In some cases, these warrant an explanation to the SIEM team (and maybe some more docs.)

'RouteCapture'
'SubstateMiddlewareFactory'
? other stuff

Continue adding doc comments to exports.

This makes the code much easier to read and understand. IDEs can show these docs on hover. Let's make it easy to read and understand our code.

Document the technical roadmap for Resolver, specifically as it relates to 'Is it an Embeddable?'

The last time the broader team heard details about the technical roadmap of Resolver, 'embeddable' was said many times. Let's explain to the team what our current vision for resolver is.

…o-siem

XavierM · 2020-05-19T13:16:54Z

@elasticmachine merge upstream

jportner

Security integration test changes (x-pack/test/api_integration/apis/security/privileges.ts, x-pack/test/api_integration/apis/security/privileges_basic.ts) LGTM.

jportner · 2020-05-19T13:51:32Z

x-pack/plugins/siem/server/config.ts

+  /**
+   * Host Endpoint Configuration
+   */
+  endpointResultListDefaultFirstPageIndex: schema.number({ defaultValue: 0 }),
+  endpointResultListDefaultPageSize: schema.number({ defaultValue: 10 }),
+
+  /**
+   * Alert Endpoint Configuration
+   */
+  alertResultListDefaultDateRange: schema.object({
+    from: schema.string({ defaultValue: 'now-15m' }),
+    to: schema.string({ defaultValue: 'now' }),
+  }),


Nit: I happened to look for the docs for these settings, and discovered that there aren't any docs for these settings (or other existing siem plugin settings). If there's not already an open issue for it, perhaps one should be opened to ensure that docs are added for these settings.

Thank you @jportner, I think we are going to revisit some of that and see if we still need it and if so we will document it

@XavierM @jportner Here's an existing github issue concerning the removal of these options: https://github.com/elastic/endpoint-app-team/issues/176

kibanamachine · 2020-05-19T15:23:22Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 3636555

History

💔 Build #48880 failed c6ad254
💔 Build #48738 failed 1c628a1
💔 Build #48699 failed 8dc39fc

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

tonymeehan · 2020-05-19T15:29:32Z

WOOOOO

* adds the stuff * keeps moving stuff * finishes moving the stuff * moves tests * fix type * try moving it all at once. BROKEN * move endpoint to siem * fix package coming from endpoint * missing scripts + change url * fix eslint * temporary disable functional testing for endpoint * fix api integration types * allow api integration test + comment functional test * fix internationalization * fix internationalization II * fix jest test * fix x-pack test * fix i18n * fix api integration * fix circular dependency * add new dependency to cypress test Co-authored-by: Davis Plumlee <davis.plumlee@elastic.co> Co-authored-by: oatkiller <robert.austin@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* [SIEMDPOINT] Move endpoint to siem (#66907) * adds the stuff * keeps moving stuff * finishes moving the stuff * moves tests * fix type * try moving it all at once. BROKEN * move endpoint to siem * fix package coming from endpoint * missing scripts + change url * fix eslint * temporary disable functional testing for endpoint * fix api integration types * allow api integration test + comment functional test * fix internationalization * fix internationalization II * fix jest test * fix x-pack test * fix i18n * fix api integration * fix circular dependency * add new dependency to cypress test Co-authored-by: Davis Plumlee <davis.plumlee@elastic.co> Co-authored-by: oatkiller <robert.austin@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> * miss import in cherry-pick Co-authored-by: Davis Plumlee <davis.plumlee@elastic.co> Co-authored-by: oatkiller <robert.austin@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* master: (33 commits) [Saved Objects] adds support for including hidden types in saved objects client (elastic#66879) [Discover] Deangularize timechart header (elastic#66532) [Discover] Improve and unskip a11y context view test (elastic#66959) [SIEM] Refactor Timeline.timelineType draft to Timeline.status draft (elastic#66864) docs: update RUM documentation link (elastic#67042) [QA] fixup coverage ingestion tests. (elastic#66905) [Metrics UI] Add support for multiple groupings to Metrics Explorer (and Alerts) (elastic#66503) [Metrics UI] Add sorting for name and value to Inventory View (elastic#66644) [Metrics UI] Change Metric Threshold Alert charts to use bar charts (elastic#66672) [Uptime] Use React.lazy for alert type registration (elastic#66829) [Reporting] Consolidate API Integration Test configs (elastic#66637) Allow histogram fields in average and sum aggregations (elastic#66891) Fix saved object share link (elastic#66771) move role reset into the top level after clause (elastic#66971) Automate the labels for any PRs affecting files for the Ingest Management team (elastic#67022) [SIEMDPOINT] Move endpoint to siem (elastic#66907) server.uuid so is not used (elastic#66963) Revert "[ci/stats] fix git metadata collection (elastic#66840)" [Uptime] Unmount uptime app properly (elastic#66950) [Visualize] Bar chart: Show missing values on chart setting (elastic#66375) ...

elasticmachine · 2021-09-23T14:34:26Z

Pinging @elastic/security-solution (Team: SecuritySolution)

dplumlee and others added 10 commits May 13, 2020 11:32

adds the stuff

99fbded

keeps moving stuff

331699d

finishes moving the stuff

75a8faa

moves tests

aac17a9

fix type

013bb40

try moving it all at once. BROKEN

0dd84c1

move endpoint to siem

46465ff

Merge branch 'master' of github.com:elastic/kibana into move-alerts-t…

9cfaaa5

…o-siem

fix package coming from endpoint

821a23a

missing scripts + change url

8dc39fc

XavierM requested review from a team as code owners May 18, 2020 16:10

XavierM added release_note:skip Skip the PR/issue when compiling release notes v7.9.0 v8.0.0 labels May 18, 2020

XavierM self-assigned this May 18, 2020

XavierM added Team:Endpoint Data Visibility Team managing the endpoint resolver Team:Endpoint Management Team:Endpoint Response Endpoint Response Team Team:SIEM labels May 18, 2020

XavierM changed the title ~~[SIENDPOINT] Move endpoint to siem~~ [SIEMDPOINT] Move endpoint to siem May 18, 2020

fix eslint

88e5997

oatkiller approved these changes May 18, 2020

View reviewed changes

temporary disable functional testing for endpoint

1c628a1

XavierM force-pushed the move-alerts-to-siem branch from 55d0615 to 1c628a1 Compare May 18, 2020 18:11

dplumlee approved these changes May 18, 2020

View reviewed changes

XavierM added 2 commits May 18, 2020 15:29

fix api integration types

1787b24

Merge branch 'master' of github.com:elastic/kibana into move-alerts-t…

1c21b76

…o-siem

peluja1012 approved these changes May 18, 2020

View reviewed changes

allow api integration test + comment functional test

8f7f736

XavierM force-pushed the move-alerts-to-siem branch from 3dd2d8f to 8f7f736 Compare May 18, 2020 20:46

XavierM added 5 commits May 18, 2020 19:34

Merge branch 'master' of github.com:elastic/kibana into move-alerts-t…

cc0e580

…o-siem

fix internationalization

a9e2463

fix internationalization II

cd0f025

fix jest test

afe33f4

fix x-pack test

b4867c3

XavierM requested a review from a team as a code owner May 18, 2020 23:49

XavierM added 2 commits May 18, 2020 20:18

Merge branch 'master' of github.com:elastic/kibana into move-alerts-t…

c0a2920

…o-siem

fix i18n

61738b0

XavierM force-pushed the move-alerts-to-siem branch from 3f532f5 to 61738b0 Compare May 19, 2020 01:46

XavierM added 3 commits May 18, 2020 22:25

fix api integration

37c771f

fix circular dependency

29cebe2

add new dependency to cypress test

c6ad254

Merge branch 'master' into move-alerts-to-siem

3636555

jportner approved these changes May 19, 2020

View reviewed changes

XavierM merged commit bcfc029 into elastic:master May 19, 2020

XavierM mentioned this pull request May 19, 2020

[7.x] [SIEMDPOINT] Move endpoint to siem (#66907) #67035

Merged

MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SIEMDPOINT] Move endpoint to siem #66907

[SIEMDPOINT] Move endpoint to siem #66907

XavierM commented May 18, 2020 •

edited

elasticmachine commented May 18, 2020

elasticmachine commented May 18, 2020

elasticmachine commented May 18, 2020

elasticmachine commented May 18, 2020

oatkiller commented May 18, 2020 •

edited

XavierM commented May 19, 2020

jportner left a comment

jportner May 19, 2020

XavierM May 19, 2020

oatkiller May 26, 2020

kibanamachine commented May 19, 2020

tonymeehan commented May 19, 2020

elasticmachine commented Sep 23, 2021

[SIEMDPOINT] Move endpoint to siem #66907

[SIEMDPOINT] Move endpoint to siem #66907

Conversation

XavierM commented May 18, 2020 • edited

Summary

Add in kibana dev config

New routes:

elasticmachine commented May 18, 2020

elasticmachine commented May 18, 2020

elasticmachine commented May 18, 2020

elasticmachine commented May 18, 2020

oatkiller commented May 18, 2020 • edited

Action items:

Re-enable and fix tests

Consolidate ‘StartDeps’.

UI ‘links’ are implemented in a different way in SIEM vs Endpoint.

Consider formalizing (and finishing) the 'Subplugin' idea

Come to agreement on eslint rules that aren’t part of the Kibana standard.

Explain certain Endpoint code concepts to the SIEM team

Continue adding doc comments to exports.

Document the technical roadmap for Resolver, specifically as it relates to 'Is it an Embeddable?'

XavierM commented May 19, 2020

jportner left a comment

Choose a reason for hiding this comment

jportner May 19, 2020

Choose a reason for hiding this comment

XavierM May 19, 2020

Choose a reason for hiding this comment

oatkiller May 26, 2020

Choose a reason for hiding this comment

kibanamachine commented May 19, 2020

💚 Build Succeeded

History

tonymeehan commented May 19, 2020

elasticmachine commented Sep 23, 2021

XavierM commented May 18, 2020 •

edited

oatkiller commented May 18, 2020 •

edited