Add elastic/enterprise-search-server service account #83325

ioanatia · 2022-01-31T12:29:45Z

Add an elastic/enterprise-search-server service account similar to elastic/kibana and elastic/fleet-server.

Still WIP because I might need to adapt some tests and also need to test it a bit more with Enterprise Search.

ywangd · 2022-01-31T23:19:47Z

...ity/src/main/java/org/elasticsearch/xpack/security/authc/service/ElasticServiceAccounts.java

+        "enterprise-search-server",
+        new RoleDescriptor(
+            NAMESPACE + "/enterprise-search-server",
+            new String[] { "manage", "manage_ilm", "manage_index_templates", "manage_security", "monitor" },


manage_security is a very powerful privilege which basically means the account has unbounded privilege. Is it necessary for enterprise search's use case?

Enterprise Search needs to be able to create/modify native users and Elasticsearch roles.
Is there a different privilege that could be used for this?

Thanks for clarification. In this case, manage_security is the privilege to go. We don't have separate privileges for working with users and roles only.

I understand this must be how Enterprise Search always works. But it would be great if we could learn about its details and potentially work out something that allows us moving away from manage_security in future.

Another point is about the manage privilege, it covers all manage_ilm, manage_index_templates and monitor (but not manage_security). It is also a powerful privilege. What actions do Enterprise Search need it for?

manage_ilm - Enterprise Search creates several data streams for logging events, for example analytics events or API logs. These data streams have an ILM policy that is created by Enterprise Search.

manage_index_templates - we use composable index templates for these data streams to ensure that they are ECS compatible

monitor - Enterprise Search needs cluster information, such as the cluster health and node information. For example, when Enterprise Search starts, it waits for the cluster to be healthy. Node information is useful to determine what plugins are available, for example we use analysis-icu if it is available.

I understand this must be how Enterprise Search always works. But it would be great if we could learn about its details and potentially work out something that allows us moving away from manage_security in future.

I agree. Is there an issue if we remove this cluster privilege for the service account some time in future when Enterprise Search no longer needs it? I don't think it is, but just asking to be sure.

I agree. Is there an issue if we remove this cluster privilege for the service account some time in future when Enterprise Search no longer needs it? I don't think it is, but just asking to be sure.

If we don't have Enterprise Search on older versions talking to ES on new versions with reduced privieges, it should not be an issue.

@ioanatia Thanks for clarifing the intended usage for manage_ilm, manage_index_templates and monitor. Do they already cover all that is necessary for Enterprise Search? Sorry if it wasn't clear, but my original question was whether we can drop manage if there is nothing else required outside of the three more fine-grained privileges. Examples of extra privileges granted by manage are manage_ml, manage_watcher, manage_slm, manage_pipeline etc. Do Enterprise Searh need any of these extra privileges?

ywangd · 2022-01-31T23:33:09Z

...ity/src/main/java/org/elasticsearch/xpack/security/authc/service/ElasticServiceAccounts.java

+                    .privileges(
+                        "create",
+                        "create_index",
+                        "delete",
+                        "delete_index",
+                        "index",
+                        "manage",
+                        "manage_ilm",
+                        "monitor",
+                        "read",
+                        "view_index_metadata",
+                        "write"
+                    )


The manage privilege covers create_index, delete_index, manage_ilm, monitor and view_index_metadata. The write privilege covers create, delete and index. So this list can be simplied down to ["read", "write", "manage"].

Above privileges combined are very close to the all privilege. The only actions that are not granted are proxy CCS related ones, e.g. internal:transport/proxy/indices:data/read/*. Is CCS something enterprise search needs?

Is CCS something enterprise search needs?

I don't think we need it at this stage, so we can just change this to ["read", "write", "manage"] as you suggested.

elasticmachine · 2022-02-04T13:17:11Z

Pinging @elastic/es-security (Team:Security)

ywangd · 2022-02-06T23:57:15Z

The v7 REST compat test fails because there are now 3 service accounts instead of 2. This particular yaml test was not well written to be friendly for REST compat test. For now, I suggest that we skip it. I raised #83564 and will fix it with a separate PR. You can skip the test by apply the following patch. Thanks!

diff --git a/x-pack/plugin/build.gradle b/x-pack/plugin/build.gradle
index ce010eae6b3..1ec97158200 100644
--- a/x-pack/plugin/build.gradle
+++ b/x-pack/plugin/build.gradle
@@ -113,6 +113,7 @@ tasks.named("yamlRestTestV7CompatTransform").configure{ task ->
   task.skipTest("indices.freeze/20_stats/Translog stats on frozen indices", "#70192 -- the freeze index API is removed from 8.0")
   task.skipTest("indices.freeze/10_basic/Basic", "#70192 -- the freeze index API is removed from 8.0")
   task.skipTest("indices.freeze/10_basic/Test index options", "#70192 -- the freeze index API is removed from 8.0")
+  task.skipTest("service_accounts/10_basic/Test get service accounts", "new service accounts are added")
 
   task.replaceValueInMatch("_type", "_doc")
   task.addAllowedWarningRegex("\\[types removal\\].*")

ywangd

LGTM

Also answer my own question: I noticed that the test asserts privilege for updating cluster setting, which requires the manage cluster privilege.

ywangd · 2022-02-08T04:08:51Z

...rc/test/java/org/elasticsearch/xpack/security/authc/service/ElasticServiceAccountsTests.java

+        // manage
+        assertThat(role.cluster().check(ClusterHealthAction.NAME, request, authentication), is(true));


Nit: cluster health is covered by the monitor privilege.

elasticsearchmachine · 2022-02-11T12:21:55Z

Hi @ioanatia, I've created a changelog YAML for you.

elasticsearchmachine added v8.1.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Jan 31, 2022

Add elastic/enterprise-search-server service account

4a2a37a

ioanatia force-pushed the enterprise_search_service_account branch from aca19ed to 4a2a37a Compare January 31, 2022 14:08

ywangd reviewed Jan 31, 2022

View reviewed changes

ioanatia added 3 commits February 1, 2022 13:02

Remove overlapping index privileges

a6dd8f3

Linting

3bbf651

Remove cluster privilege already covered by manage

a8e9239

ioanatia changed the title ~~[WIP] Add elastic/enterprise-search-server service account~~ Add elastic/enterprise-search-server service account Feb 2, 2022

mark-vieira added v8.2.0 and removed v8.1.0 labels Feb 2, 2022

Merge branch 'master' into enterprise_search_service_account

ecee090

ioanatia marked this pull request as ready for review February 4, 2022 13:15

ioanatia added the Team:Security Meta label for security team label Feb 4, 2022

ywangd added the :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC label Feb 6, 2022

ywangd added the >enhancement label Feb 6, 2022

Skip test

c54ada6

ioanatia force-pushed the enterprise_search_service_account branch from a1d262e to c54ada6 Compare February 7, 2022 13:51

ioanatia requested a review from ywangd February 7, 2022 15:14

ywangd approved these changes Feb 8, 2022

View reviewed changes

Reorder assertions in test

4c3f7c7

ioanatia and others added 2 commits February 11, 2022 13:21

Update docs/changelog/83325.yaml

f1eb9d1

Merge branch 'master' into enterprise_search_service_account

d435f65

ioanatia merged commit 8487b03 into elastic:master Feb 11, 2022

ioanatia deleted the enterprise_search_service_account branch February 11, 2022 14:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add elastic/enterprise-search-server service account #83325

Add elastic/enterprise-search-server service account #83325

ioanatia commented Jan 31, 2022

ywangd Jan 31, 2022

ioanatia Feb 1, 2022

ywangd Feb 1, 2022

ywangd Feb 1, 2022

ioanatia Feb 2, 2022

ywangd Feb 4, 2022

ywangd Feb 6, 2022

ywangd Jan 31, 2022

ioanatia Feb 1, 2022

elasticmachine commented Feb 4, 2022

ywangd commented Feb 6, 2022

ywangd left a comment

ywangd Feb 8, 2022

elasticsearchmachine commented Feb 11, 2022

		// manage
		assertThat(role.cluster().check(ClusterHealthAction.NAME, request, authentication), is(true));

Add elastic/enterprise-search-server service account #83325

Add elastic/enterprise-search-server service account #83325

Conversation

ioanatia commented Jan 31, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Feb 4, 2022

ywangd commented Feb 6, 2022

ywangd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticsearchmachine commented Feb 11, 2022