[ML] Adding endpoint capability checks #64662

jgowdyelastic · 2020-04-28T14:59:19Z

Adds capabilities checks to all of our kibana endpoints.
Introduces a canAccessML capability to cover access to the ML app on basic or above licenses.

Part of #64172

Unit or functional tests were updated or added to match the most common scenarios
This was checked for breaking API changes and was labeled appropriately

elasticmachine · 2020-04-28T17:59:14Z

Pinging @elastic/ml-ui (:ml)

jgowdyelastic · 2020-04-28T19:45:25Z

@elasticmachine merge upstream

walterra

LGTM

…elastic/kibana into adding-endpoint-capability-checks

x-pack/plugins/ml/server/routes/annotations.ts

x-pack/plugins/ml/server/routes/datafeeds.ts

x-pack/plugins/ml/server/routes/job_service.ts

peteharverson · 2020-04-29T13:56:22Z

x-pack/plugins/ml/server/routes/system.ts

@@ -150,7 +156,11 @@ export function systemRoutes(
    {
      path: '/api/ml/ml_node_count',
      validate: false,
+      options: {
+        tags: ['access:ml:canGetJobs'],


Just curious as to why this isn't canAccessML like the other routes in here?

canAccessML means it's also available on basic.
An end user shouldn't need to query the ml node count if they only have a basic license.

alvarezmelissa87

LGTM ⚡

kibanamachine · 2020-04-29T16:29:54Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: a9f0df1

History

💔 Build #44039 failed 191cfad
💔 Build #43846 failed a92d6e3
💔 Build #43790 failed 0a82531

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

peteharverson

Latest edits LGTM.

Don't think this is related to these changes, but could we be hiding the ML Jobs List item from the Management page for the 'user' role? It is shown currently, but clicking on it gives the ML access_denied page

Also the 'Clone job' link for the ML viewer on the data frame analytics jobs page is still enabled.

* [ML] Adding endpoint capability checks * adding missing capability checks * fixing test * removing commented code * fixing functional test * fixing functional tests * changes based on review Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* [ML] Adding endpoint capability checks * adding missing capability checks * fixing test * removing commented code * fixing functional test * fixing functional tests * changes based on review Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

…bana into pipeline-editor-part-mvp-2 * 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (90 commits) remove unused import address review feedback [Ingest pipelines] Cleanup (elastic#64794) [Ingest] Edit datasource UI (elastic#64727) [Lens] Bind all time fields to the time picker (elastic#63874) [Lens] Use suggestion system in chart switcher for subtypes (elastic#64613) Improve alpha messaging (elastic#64692) [Ingest] Allow to enable monitoring of elastic agent (elastic#63598) [Metrics UI] Fix alerting when a filter query is present (elastic#64575) skip flaky suite (elastic#64812) (elastic#64723) [Maps] do not display EMS or kibana layer wizards when not configured (elastic#64554) [Reporting/Test] Convert functional test code to Typescript (elastic#64601) make inserting timestamp with navigate methods optional with default true (elastic#64655) [EPM] Update UI to handle package versions and updates (elastic#64689) Minimize dependencies required by our telemetry middleware (elastic#64665) [Telemetry] oss api tests (elastic#64602) [ML] Adding endpoint capability checks (elastic#64662) Update jest config for coverage (elastic#64648) [SIEM][NP] Fixes bug in ML signals promotion (elastic#64720) share single data plugin bundle (elastic#64549) ...

…ssions (#74582) ## Summary Addresses #73567. ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of #64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`. To solve this I've created two separate hooks to replace `useSiemJobs`: * `useSecurityJobs` * used on ML Popover * includes uninstalled ML Jobs * checks (and returns) `isMlAdmin` before fetching data * `useInstalledSecurityJobs` * used on ML Jobs Dropdown and Anomalies Table * includes only installed ML Jobs * checks (and returns) `isMlUser` before fetching data Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors: #### User has insufficient license * ML Popover: shows an upgrade CTA * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled, shows upgrade CTA * Rule Details: ML Job Id is displayed as text #### User is ML User * ML Popover: not shown * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled * Rule Details: ML Job Id is displayed as text #### User is ML Admin * ML Popover: shown * Anomalies Tables: show data __for installed ML Jobs__ * This is the same as previous logic, but worth calling out that you can't view historical anomalies * Rule Creation: ML Rule option is enabled, all ML Jobs available * Rule Details: ML Job Id is displayed as hyperlink, job status badge shown ### Checklist - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

…ssions (elastic#74582) ## Summary Addresses elastic#73567. ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of elastic#64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`. To solve this I've created two separate hooks to replace `useSiemJobs`: * `useSecurityJobs` * used on ML Popover * includes uninstalled ML Jobs * checks (and returns) `isMlAdmin` before fetching data * `useInstalledSecurityJobs` * used on ML Jobs Dropdown and Anomalies Table * includes only installed ML Jobs * checks (and returns) `isMlUser` before fetching data Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors: #### User has insufficient license * ML Popover: shows an upgrade CTA * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled, shows upgrade CTA * Rule Details: ML Job Id is displayed as text #### User is ML User * ML Popover: not shown * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled * Rule Details: ML Job Id is displayed as text #### User is ML Admin * ML Popover: shown * Anomalies Tables: show data __for installed ML Jobs__ * This is the same as previous logic, but worth calling out that you can't view historical anomalies * Rule Creation: ML Rule option is enabled, all ML Jobs available * Rule Details: ML Job Id is displayed as hyperlink, job status badge shown ### Checklist - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

…ssions (#74582) (#74919) ## Summary Addresses #73567. ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of #64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`. To solve this I've created two separate hooks to replace `useSiemJobs`: * `useSecurityJobs` * used on ML Popover * includes uninstalled ML Jobs * checks (and returns) `isMlAdmin` before fetching data * `useInstalledSecurityJobs` * used on ML Jobs Dropdown and Anomalies Table * includes only installed ML Jobs * checks (and returns) `isMlUser` before fetching data Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors: #### User has insufficient license * ML Popover: shows an upgrade CTA * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled, shows upgrade CTA * Rule Details: ML Job Id is displayed as text #### User is ML User * ML Popover: not shown * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled * Rule Details: ML Job Id is displayed as text #### User is ML Admin * ML Popover: shown * Anomalies Tables: show data __for installed ML Jobs__ * This is the same as previous logic, but worth calling out that you can't view historical anomalies * Rule Creation: ML Rule option is enabled, all ML Jobs available * Rule Details: ML Job Id is displayed as hyperlink, job status badge shown ### Checklist - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process) Co-authored-by: Ryland Herrick <ryalnd@gmail.com>

…ssions (elastic#74582) ## Summary Addresses elastic#73567. ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of elastic#64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`. To solve this I've created two separate hooks to replace `useSiemJobs`: * `useSecurityJobs` * used on ML Popover * includes uninstalled ML Jobs * checks (and returns) `isMlAdmin` before fetching data * `useInstalledSecurityJobs` * used on ML Jobs Dropdown and Anomalies Table * includes only installed ML Jobs * checks (and returns) `isMlUser` before fetching data Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors: #### User has insufficient license * ML Popover: shows an upgrade CTA * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled, shows upgrade CTA * Rule Details: ML Job Id is displayed as text #### User is ML User * ML Popover: not shown * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled * Rule Details: ML Job Id is displayed as text #### User is ML Admin * ML Popover: shown * Anomalies Tables: show data __for installed ML Jobs__ * This is the same as previous logic, but worth calling out that you can't view historical anomalies * Rule Creation: ML Rule option is enabled, all ML Jobs available * Rule Details: ML Job Id is displayed as hyperlink, job status badge shown ### Checklist - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

…ssions (#74582) (#75287) ## Summary Addresses #73567. ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of #64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`. To solve this I've created two separate hooks to replace `useSiemJobs`: * `useSecurityJobs` * used on ML Popover * includes uninstalled ML Jobs * checks (and returns) `isMlAdmin` before fetching data * `useInstalledSecurityJobs` * used on ML Jobs Dropdown and Anomalies Table * includes only installed ML Jobs * checks (and returns) `isMlUser` before fetching data Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors: #### User has insufficient license * ML Popover: shows an upgrade CTA * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled, shows upgrade CTA * Rule Details: ML Job Id is displayed as text #### User is ML User * ML Popover: not shown * Anomalies Tables: show no data * Rule Creation: ML Rule option is disabled * Rule Details: ML Job Id is displayed as text #### User is ML Admin * ML Popover: shown * Anomalies Tables: show data __for installed ML Jobs__ * This is the same as previous logic, but worth calling out that you can't view historical anomalies * Rule Creation: ML Rule option is enabled, all ML Jobs available * Rule Details: ML Job Id is displayed as hyperlink, job status badge shown ### Checklist - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process) Co-authored-by: Ryland Herrick <ryalnd@gmail.com>

jgowdyelastic added 3 commits April 28, 2020 15:57

[ML] Adding endpoint capability checks

0a82531

adding missing capability checks

de1f763

fixing test

5516436

jgowdyelastic requested review from alvarezmelissa87, peteharverson, darnautov and qn895 April 28, 2020 17:58

jgowdyelastic self-assigned this Apr 28, 2020

jgowdyelastic added :ml non-issue Indicates to automation that a pull request should not appear in the release notes release_note:skip Skip the PR/issue when compiling release notes review labels Apr 28, 2020

jgowdyelastic added v7.8.0 v8.0.0 labels Apr 28, 2020

jgowdyelastic marked this pull request as ready for review April 28, 2020 17:59

jgowdyelastic requested a review from a team as a code owner April 28, 2020 17:59

removing commented code

a92d6e3

Merge branch 'master' into adding-endpoint-capability-checks

ba394c1

walterra approved these changes Apr 29, 2020

View reviewed changes

jgowdyelastic added 3 commits April 29, 2020 12:33

fixing functional test

2909a93

Merge branch 'adding-endpoint-capability-checks' of github.com:jgowdy…

191cfad

…elastic/kibana into adding-endpoint-capability-checks

fixing functional tests

d70f40d

peteharverson reviewed Apr 29, 2020

View reviewed changes

changes based on review

a9f0df1

alvarezmelissa87 reviewed Apr 29, 2020

View reviewed changes

alvarezmelissa87 approved these changes Apr 29, 2020

View reviewed changes

peteharverson approved these changes Apr 29, 2020

View reviewed changes

jgowdyelastic merged commit d8c15f5 into elastic:master Apr 29, 2020

jgowdyelastic deleted the adding-endpoint-capability-checks branch April 29, 2020 17:31

jgowdyelastic mentioned this pull request Apr 29, 2020

[7.x] [ML] Adding endpoint capability checks (#64662) #64811

Merged

jgowdyelastic mentioned this pull request Apr 29, 2020

[ML] Integration with Kibana Spaces #64172

Closed

26 tasks

peteharverson mentioned this pull request May 5, 2020

[ML] Restrict get module and recognizer endpoints #61223

Closed

spong mentioned this pull request Jul 28, 2020

[Security Solution][Detections] SIEM job fetch failure error toast shown for users without ML permissions #73567

Closed

rylnd mentioned this pull request Aug 6, 2020

[Security Solution][Detections] Refactor ML calls for newest ML permissions #74582

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Adding endpoint capability checks #64662

[ML] Adding endpoint capability checks #64662

jgowdyelastic commented Apr 28, 2020 •

edited by peteharverson

Loading

elasticmachine commented Apr 28, 2020

jgowdyelastic commented Apr 28, 2020

walterra left a comment

peteharverson Apr 29, 2020

jgowdyelastic Apr 29, 2020

alvarezmelissa87 left a comment

kibanamachine commented Apr 29, 2020

peteharverson left a comment

[ML] Adding endpoint capability checks #64662

[ML] Adding endpoint capability checks #64662

Conversation

jgowdyelastic commented Apr 28, 2020 • edited by peteharverson Loading

elasticmachine commented Apr 28, 2020

jgowdyelastic commented Apr 28, 2020

walterra left a comment

Choose a reason for hiding this comment

peteharverson Apr 29, 2020

Choose a reason for hiding this comment

jgowdyelastic Apr 29, 2020

Choose a reason for hiding this comment

alvarezmelissa87 left a comment

Choose a reason for hiding this comment

kibanamachine commented Apr 29, 2020

💚 Build Succeeded

History

peteharverson left a comment

Choose a reason for hiding this comment

jgowdyelastic commented Apr 28, 2020 •

edited by peteharverson

Loading