Index Pattern UI - remove _remote/info query #27345

mattkime · 2018-12-17T21:38:03Z

Summary

Index Pattern UI - remove _remote/info query and replace with search for *:* which we can also use for determining whether there are remote clusters

Resolves "kibana_user sees "Failed to load remote clusters" when creating index pattern" - #27093

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~[ ] This was checked for cross-browser compatibility, including a check against IE11~~
~~[ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support~~
~~[ ] Documentation was added for features that require explanation or tutorials~~
~~[ ] Unit or functional tests were updated or added to match the most common scenarios~~
~~[ ] This was checked for keyboard-only and screenreader accessibility~~

For maintainers

~~[ ] This was checked for breaking API changes and was labeled appropriately~~
~~[ ] This includes a feature addition or change that requires a release note and was labeled appropriately~~

elasticmachine · 2018-12-17T21:42:41Z

Pinging @elastic/kibana-app

elasticmachine · 2018-12-17T21:42:49Z

Pinging @elastic/kibana-security

kobelb · 2018-12-17T21:48:04Z

Which privilege is necessary to use this endpoint? Are we sure that the kibana server user will have this?

mattkime · 2018-12-17T21:56:33Z

@kobelb We can expect the user to have kibana_user, although I suspect I haven't implemented this. I could use some guidance.

Are we sure that the kibana server user will have this?

Uh, worked for me. But perhaps you know of a more rigorous method of determining the answer.

elasticmachine · 2018-12-17T22:17:48Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

legrego · 2018-12-18T13:24:47Z

We can expect the user to have kibana_user, although I suspect I haven't implemented this. I could use some guidance.

We can't assume that users will have the kibana_user role anymore, with the adoption of Kibana's new authorization model ("application privileges").

Based on my testing, it appears that the _remote/info endpoint requires the monitor cluster privilege. The kibana_system role (which the callWithInternalUser account has) is granted monitor, so that should be sufficient:

mattkime · 2018-12-18T15:11:13Z

@legrego

We can't assume that users will have the kibana_user role anymore, with the adoption of Kibana's new authorization model ("application privileges").

What privileges can we assume a basic user will have?

kobelb · 2018-12-18T15:16:32Z

Based on my testing, it appears that the _remote/info endpoint requires the monitor cluster privilege. The kibana_system role (which the callWithInternalUser account has) is granted monitor, so that should be sufficient:

Thanks for researching this Larry. So, at this point we're in a situation where the greatest risk to using callWithInternalUser is potentially informing a user indirectly of the existence of a remote cluster, which they wouldn't have permission to see otherwise. How important is changing the UI to denote this situation?

What privileges can we assume a basic user will have?

We should be assuming that users with access to Kibana only have access to Kibana using the ES application privileges: https://www.elastic.co/guide/en/elastic-stack-overview/6.5/security-privileges.html#application-privileges

These do not grant them any additional access to ES APIs, and only indirectly grant them access to Kibana which Kibana enforces.

legrego · 2018-12-18T15:17:29Z

It's possible that a Kibana end user will only have Kibana application privileges defined. It's technically possible for a user to access Kibana without any cluster or index privileges, although it'd be of limited use.

Users will typically have Kibana application privileges, and then privileges to their own data indices (not necessarily system indices). Certain cluster/index privileges are required to enable certain features within Kibana, but they aren't required to use Kibana in general.

kobelb · 2018-12-18T15:17:45Z

@mattkime, the following might be helpful: https://www.elastic.co/guide/en/kibana/current/development-security.html

mattkime · 2018-12-18T16:48:07Z

I'm going to discuss this with design. The design does not lend itself to 'we don't have access to determine if you have any indexes'

mattkime · 2018-12-19T23:32:38Z

Refactored to only use search api.

@kobelb might be interested

elasticmachine · 2018-12-20T00:09:36Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-12-20T03:19:26Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-12-20T03:45:58Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

LeeDr

Works for me.

I tested by copying the file /src/legacy/core_plugins/kibana/public/management/sections/index_patterns/create_index_pattern_wizard/create_index_pattern_wizard.js from this PR to my 7.0.0-alpha2 install, removed everything from my optimize directory and restarted Kibana.

Now as the same user as originally described in the issue, I can create index patterns (both local and cross cluster) without the error. So I think this fix is good.

There is a different problem in this same area but I don't think this PR caused it or fixes it. The problem is if a normal kibana_user enters something like logstash- and removes the trailing *, then Kibana just churns and churns on Looking for matching indices

While the console shows;

Detected an unhandled Promise rejection.
[security_exception] action [indices:admin/shards/search_shards] is unauthorized for user [power] :: {"path":"/*%3Am/_search","query":{},"body":"{\"size\":0,\"aggs\":{\"indices\":{\"terms\":{\"field\":\"_index\",\"size\":200}}}}","statusCode":403,"response":"{\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"action [indices:admin/shards/search_shards] is unauthorized for user [power]\"}],\"type\":\"security_exception\",\"reason\":\"action [indices:admin/shards/search_shards] is unauthorized for user [power]\"},\"status\":403}"}
vendors.bundle.dll.js:95 Uncaught (in promise) StatusCodeError {status: 403, displayName: "AuthorizationException", message: "[security_exception] action [indices:admin/shards/search_shards] is unauthorized for user [power]", path: "/*%3Am/_search", query: {…}, …}
step @ kibana.bundle.js:2
(anonymous) @ kibana.bundle.js:2
Promise.then (async)
step @ kibana.bundle.js:2
(anonymous) @ kibana.bundle.js:2
(anonymous) @ kibana.bundle.js:2
fetchIndices @ kibana.bundle.js:2
onQueryChanged @ kibana.bundle.js:2
ka @ vendors.bundle.dll.js:203
invokeGuardedCallback @ vendors.bundle.dll.js:203
invokeGuardedCallbackAndCatchFirstError @ vendors.bundle.dll.js:203
Fa @ vendors.bundle.dll.js:203
Ja @ vendors.bundle.dll.js:203
La @ vendors.bundle.dll.js:203
Ha @ vendors.bundle.dll.js:203
Oa @ vendors.bundle.dll.js:203
Pa @ vendors.bundle.dll.js:203
Td @ vendors.bundle.dll.js:203
batchedUpdates @ vendors.bundle.dll.js:203
dc @ vendors.bundle.dll.js:203
Xd @ vendors.bundle.dll.js:203
interactiveUpdates @ vendors.bundle.dll.js:203
Wd @ vendors.bundle.dll.js:203
VM6529:1 POST https://localhost:5601/elasticsearch/*%3Ama/_search 403 (Forbidden)

I'm not sure when this issue was introduced. I'll try to figure that out and file an issue on it.

elasticmachine · 2018-12-20T21:37:43Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

mattkime · 2018-12-20T22:30:14Z

...ana/public/management/sections/index_patterns/create_index_pattern_wizard/lib/get_indices.js

@@ -46,6 +46,7 @@ export async function getIndices(es, indexPatternCreationType, rawPattern, limit
  }

  const params = {
+    ignoreUnavailable: true,


This addresses @LeeDr 's concern #27345 (review)

Requests without wildcards that failed to match were returning 500 errors. It seems that they should be returning 404 errors but its easier to apply this fix than track that down.

Its an unrelated bug but I thought I might as well address it here.

same issue is described here - #27218

elasticmachine · 2018-12-20T23:03:23Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

LeeDr · 2018-12-21T21:22:43Z

darn pie chart test! retest

LeeDr · 2018-12-21T21:23:03Z

jenkins test this

elasticmachine · 2018-12-21T21:55:25Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

lukeelmers

Code LGTM. Added two low-priority comments, but nothing that should block this from being merged.

I haven't yet been able to test locally, but since @LeeDr already has, I will approve now so we can get this in.

lukeelmers · 2019-01-04T00:04:20Z

...anagement/sections/index_patterns/create_index_pattern_wizard/create_index_pattern_wizard.js

-  }
+    this.catchAndWarn(
+      // if we get an error from remote cluster query, supply fallback value that allows user entry
+      getIndices(services.es, this.indexPatternCreationType, `*:*`, 1), ['a'], clustersFailMsg


I'm a little confused by the ['a'] argument to catchAndWarn here... Does it serve a purpose, or is it just an arbitrary placeholder?

its just a placeholder value. I'll make my comment more specific

lukeelmers · 2019-01-04T00:13:42Z

...anagement/sections/index_patterns/create_index_pattern_wizard/create_index_pattern_wizard.js

+      // if we get an error from remote cluster query, supply fallback value that allows user entry
+      getIndices(services.es, this.indexPatternCreationType, `*:*`, 1), ['a'], clustersFailMsg
+    ).then(remoteIndices => this.setState({ remoteClustersExist: !!remoteIndices.length }));
+  };


nit: Any particular reason for changing the async/await in the two queries above back to Promise().then()? vs keeping the same convention, e.g.

const allIndices = await ensureMinimumTime(this.catchAndWarn(...)); this.setState({ allIndices, isInitiallyLoadingIndices: false }); const remoteIndices = await ensureMinimumTime(this.catchAndWarn(...)); this.setState({ remoteClustersExist: !!remoteIndices.length });

Ah, of course. I assume it is because you don’t care about what order those promises are resolved in, so you are letting them run asynchronously.

@lukeelmers you got it!

…:mattkime/kibana into kibana_user_remote_cluster_index_pattern

elasticmachine · 2019-01-04T04:50:48Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

* use internal user to query _remote/info * query both * and *:* rather than querying for clusters * Update create_index_pattern_wizard.js * Update create_index_pattern_wizard.js * add a closing curly bracket * fix for failed matches * Update create_index_pattern_wizard.js

use internal user to query _remote/info

8d77f7b

mattkime added the v6.6.0 label Dec 17, 2018

mattkime requested a review from kobelb December 17, 2018 21:40

mattkime added the Team:Visualizations Visualization editors, elastic-charts and infrastructure label Dec 17, 2018

mattkime added the Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! label Dec 17, 2018

mattkime added bug Fixes for quality problems that affect the customer experience regression labels Dec 17, 2018

query both * and *:* rather than querying for clusters

860f26c

mattkime changed the title ~~Index Pattern UI - use internal user to query _remote/info~~ Index Pattern UI - remove _remote/info query Dec 19, 2018

mattkime requested a review from jen-huang December 19, 2018 23:32

jen-huang added v7.0.0 v6.7.0 labels Dec 19, 2018

Update create_index_pattern_wizard.js

b68a5bc

Update create_index_pattern_wizard.js

50b4dc4

LeeDr self-requested a review December 20, 2018 04:17

LeeDr approved these changes Dec 20, 2018

View reviewed changes

mattkime added 2 commits December 20, 2018 14:58

Merge branch 'master' into kibana_user_remote_cluster_index_pattern

a2a94d0

add a closing curly bracket

3c95438

fix for failed matches

5efcea3

mattkime commented Dec 20, 2018

View reviewed changes

lukeelmers self-requested a review January 2, 2019 16:08

lukeelmers approved these changes Jan 4, 2019

View reviewed changes

mattkime added 3 commits January 3, 2019 21:46

Update create_index_pattern_wizard.js

edc5d1b

Merge branch 'master' into kibana_user_remote_cluster_index_pattern

e570ea0

Merge branch 'kibana_user_remote_cluster_index_pattern' of github.com…

f466a09

…:mattkime/kibana into kibana_user_remote_cluster_index_pattern

mattkime merged commit 20ed7d4 into elastic:master Jan 4, 2019

mattkime mentioned this pull request Jan 4, 2019

[6.x] Index Pattern UI - remove _remote/info query (#27345) #28057

Merged

mattkime mentioned this pull request Jan 4, 2019

[6.6] Index Pattern UI - remove _remote/info query (#27345) #28058

Merged

This was referenced Jan 4, 2019

AuthorizationException in index pattern wizard #27218

Closed

kibana_user sees "Failed to load remote clusters" when creating index pattern #27093

Closed

hgoscenski-vail mentioned this pull request Apr 19, 2019

Cannot Create Cross Site Index Patterns in Kibana 6.7.1 #35229

Closed

jbudz mentioned this pull request Sep 11, 2019

Kibana can’t create index pattern #45341

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Index Pattern UI - remove _remote/info query #27345

Index Pattern UI - remove _remote/info query #27345

mattkime commented Dec 17, 2018 •

edited

Loading

elasticmachine commented Dec 17, 2018

elasticmachine commented Dec 17, 2018

kobelb commented Dec 17, 2018

mattkime commented Dec 17, 2018

elasticmachine commented Dec 17, 2018

legrego commented Dec 18, 2018

mattkime commented Dec 18, 2018

kobelb commented Dec 18, 2018

legrego commented Dec 18, 2018

kobelb commented Dec 18, 2018

mattkime commented Dec 18, 2018

mattkime commented Dec 19, 2018

elasticmachine commented Dec 20, 2018

elasticmachine commented Dec 20, 2018

elasticmachine commented Dec 20, 2018

LeeDr left a comment

elasticmachine commented Dec 20, 2018

mattkime Dec 20, 2018 •

edited

Loading

mattkime Dec 21, 2018

elasticmachine commented Dec 20, 2018

LeeDr commented Dec 21, 2018

LeeDr commented Dec 21, 2018

elasticmachine commented Dec 21, 2018

lukeelmers left a comment

lukeelmers Jan 4, 2019

mattkime Jan 4, 2019

lukeelmers Jan 4, 2019

lukeelmers Jan 4, 2019

mattkime Jan 4, 2019

elasticmachine commented Jan 4, 2019

Index Pattern UI - remove _remote/info query #27345

Index Pattern UI - remove _remote/info query #27345

Conversation

mattkime commented Dec 17, 2018 • edited Loading

Summary

Checklist

For maintainers

elasticmachine commented Dec 17, 2018

elasticmachine commented Dec 17, 2018

kobelb commented Dec 17, 2018

mattkime commented Dec 17, 2018

elasticmachine commented Dec 17, 2018

💚 Build Succeeded

legrego commented Dec 18, 2018

mattkime commented Dec 18, 2018

kobelb commented Dec 18, 2018

legrego commented Dec 18, 2018

kobelb commented Dec 18, 2018

mattkime commented Dec 18, 2018

mattkime commented Dec 19, 2018

elasticmachine commented Dec 20, 2018

💚 Build Succeeded

elasticmachine commented Dec 20, 2018

💔 Build Failed

elasticmachine commented Dec 20, 2018

💔 Build Failed

LeeDr left a comment

Choose a reason for hiding this comment

elasticmachine commented Dec 20, 2018

💚 Build Succeeded

mattkime Dec 20, 2018 • edited Loading

Choose a reason for hiding this comment

mattkime Dec 21, 2018

Choose a reason for hiding this comment

elasticmachine commented Dec 20, 2018

💔 Build Failed

LeeDr commented Dec 21, 2018

LeeDr commented Dec 21, 2018

elasticmachine commented Dec 21, 2018

💚 Build Succeeded

lukeelmers left a comment

Choose a reason for hiding this comment

lukeelmers Jan 4, 2019

Choose a reason for hiding this comment

mattkime Jan 4, 2019

Choose a reason for hiding this comment

lukeelmers Jan 4, 2019

Choose a reason for hiding this comment

lukeelmers Jan 4, 2019

Choose a reason for hiding this comment

mattkime Jan 4, 2019

Choose a reason for hiding this comment

elasticmachine commented Jan 4, 2019

💚 Build Succeeded

mattkime commented Dec 17, 2018 •

edited

Loading

mattkime Dec 20, 2018 •

edited

Loading