Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand beats_system role privileges #40876

Merged

Conversation

Projects
None yet
4 participants
@ycombinator
Copy link
Contributor

ycombinator commented Apr 4, 2019

Traditionally we have recommended that Beats send their monitoring data to the production Elasticsearch cluster. Beats do this by calling the POST _monitoring/bulk API. When Security is enabled this API call requires the cluster:admin/xpack/monitoring/bulk privilege. The built-in beats_system role has this privilege.

Going forward, Beats will be able to send their monitoring data directly to the monitoring Elasticsearch cluster. Beats will do this by calling the regular POST _bulk API. When Security is enabled this API call requires the indices:data/write/bulk privilege. Further, the call has to be able to create any indices that don't exist.

This PR expands the built-in beats_system role's privileges. Specifically, it adds index-level write and create_index privileges for .monitoring-beats-* indices.

This will allow Beats users to continue using the beats_system role for the new direct monitoring route when Security is enabled.

@elasticmachine

This comment has been minimized.

Copy link

elasticmachine commented Apr 4, 2019

@elasticmachine

This comment has been minimized.

Copy link

elasticmachine commented Apr 4, 2019

@urso

This comment has been minimized.

Copy link

urso commented Apr 5, 2019

They way monitoring works and is configured in Beats, this change makes sense. It creates the least user-disruption.

What about the template/index mapping? Is it setup correctly if Beats uses the bulk API?

@ycombinator

This comment has been minimized.

Copy link
Contributor Author

ycombinator commented Apr 5, 2019

What about the template/index mapping? Is it setup correctly if Beats uses the bulk API?

The monitoring plugin in Elasticsarch sets up the monitoring index templates. So as long as the cluster that Beats is shipping to has xpack.monitoring.enabled: true (which is the default), the correct monitoring index templates will get created when the cluster is created.

new String[] { "monitor", MonitoringBulkAction.NAME},
new RoleDescriptor.IndicesPrivileges[]{
RoleDescriptor.IndicesPrivileges.builder()
.indices(".monitoring-beats-*").privileges("create_index", "write").build()

This comment has been minimized.

Copy link
@tvernum

tvernum Apr 15, 2019

Contributor

Can we make this create instead of write ?
That gives the ability to index documents (including overwriting existing documents), but not call the update or delete APIs.

I believe that's sufficient for your purposes, and would get us closer to having least privilege users & roles.

This comment has been minimized.

Copy link
@ycombinator

ycombinator Apr 15, 2019

Author Contributor

Thanks, indeed create instead of write is sufficient here as the .monitoring-beats-* indices are meant to be append-only and the create privilege allows for bulk indexing as well.

Fixed in 6bf4d8b.

is(false));
assertThat(beatsSystemRole.indices().allowedIndicesMatcher(CreateIndexAction.NAME).test(index), is(true));
assertThat(beatsSystemRole.indices().allowedIndicesMatcher(IndexAction.NAME).test(index), is(true));
assertThat(beatsSystemRole.indices().allowedIndicesMatcher(DeleteAction.NAME).test(index), is(true));

This comment has been minimized.

Copy link
@tvernum

tvernum Apr 15, 2019

Contributor

If you're happy with my suggestion above, then this will need to be false.

@ycombinator ycombinator force-pushed the ycombinator:security/beats-system-role-expand branch from 348390b to 6bf4d8b Apr 15, 2019

ycombinator added some commits Apr 4, 2019

@ycombinator ycombinator force-pushed the ycombinator:security/beats-system-role-expand branch from 811b163 to d9e4cb0 Apr 15, 2019

@ycombinator

This comment has been minimized.

Copy link
Contributor Author

ycombinator commented Apr 16, 2019

@tvernum I made the change you suggested. This is ready for your review again. Thanks!

@tvernum
Copy link
Contributor

tvernum left a comment

LGTM

@ycombinator ycombinator merged commit 64768bf into elastic:master Apr 16, 2019

8 checks passed

CLA All commits in pull request signed
Details
elasticsearch-ci/1 Build finished.
Details
elasticsearch-ci/2 Build finished.
Details
elasticsearch-ci/bwc Build finished.
Details
elasticsearch-ci/default-distro Build finished.
Details
elasticsearch-ci/docbldesx Build finished.
Details
elasticsearch-ci/oss-distro-docs Build finished.
Details
elasticsearch-ci/packaging-sample Build finished.
Details

@ycombinator ycombinator deleted the ycombinator:security/beats-system-role-expand branch Apr 16, 2019

ycombinator added a commit to ycombinator/elasticsearch that referenced this pull request Apr 16, 2019

Expand beats_system role privileges (elastic#40876)
Traditionally we have [recommended](https://www.elastic.co/guide/en/beats/filebeat/current/monitoring.html) that Beats send their monitoring data to the **production** Elasticsearch cluster. Beats do this by calling the `POST _monitoring/bulk` API. When Security is enabled this API call requires the `cluster:admin/xpack/monitoring/bulk` privilege. The built-in `beats_system` role has this privilege.

[Going forward](elastic/beats#9260), Beats will be able to send their monitoring data directly to the **monitoring** Elasticsearch cluster. Beats will do this by calling the regular `POST _bulk` API. When Security is enabled this API call requires the `indices:data/write/bulk` privilege. Further, the call has to be able to create any indices that don't exist.

This PR expands the built-in `beats_system` role's privileges. Specifically, it adds index-level `write` and `create_index` privileges for `.monitoring-beats-*` indices. 

This will allow Beats users to continue using the `beats_system` role for the new direct monitoring route when Security is enabled.

ycombinator added a commit that referenced this pull request Apr 16, 2019

Expand beats_system role privileges (#40876) (#41232)
Traditionally we have [recommended](https://www.elastic.co/guide/en/beats/filebeat/current/monitoring.html) that Beats send their monitoring data to the **production** Elasticsearch cluster. Beats do this by calling the `POST _monitoring/bulk` API. When Security is enabled this API call requires the `cluster:admin/xpack/monitoring/bulk` privilege. The built-in `beats_system` role has this privilege.

[Going forward](elastic/beats#9260), Beats will be able to send their monitoring data directly to the **monitoring** Elasticsearch cluster. Beats will do this by calling the regular `POST _bulk` API. When Security is enabled this API call requires the `indices:data/write/bulk` privilege. Further, the call has to be able to create any indices that don't exist.

This PR expands the built-in `beats_system` role's privileges. Specifically, it adds index-level `write` and `create_index` privileges for `.monitoring-beats-*` indices. 

This will allow Beats users to continue using the `beats_system` role for the new direct monitoring route when Security is enabled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.