diff --git a/docs/reference/monitoring/indices.asciidoc b/docs/reference/monitoring/indices.asciidoc index e5052a5070eb6..c9dfc424cef1a 100644 --- a/docs/reference/monitoring/indices.asciidoc +++ b/docs/reference/monitoring/indices.asciidoc @@ -1,79 +1,147 @@ [role="xpack"] [[config-monitoring-indices]] -== Configuring indices for monitoring +== Configuring data streams/indices for monitoring -Sometimes the default index settings might not work for your situation. For -example, you might want to change index lifecycle management (ILM) settings, -add custom mappings, or set the number of shards and replicas -for the monitoring indices. +Monitoring data is stored in data streams or indices in {es}. The default data +stream or index settings may not work for your situation. For example, you might +want to change index lifecycle management (ILM) settings, add custom mappings, +or change the number of shards and replicas. The steps to change these +settings depend on the monitoring method: -You can change the default behavior. The steps you follow depend on which data -collection method is used. +* <> +* <> +* <> (the default +for the {ess} on {ecloud}) -[float] +IMPORTANT: Changing mappings or settings can cause your monitoring dashboards to +stop working correctly. + +[[config-monitoring-data-streams-elastic-agent]] === Configuring data streams created by {agent} -{agent} uses data streams to store time series data across multiple indices -while giving you a single named resource for requests. You can alter the -settings of each dataset by configuring an `@custom` component template. -For more information, refer to {fleet-guide}/data-streams.html[Data streams]. +When <>, data is stored in a +set of data streams named +`metrics-{product}.stack_monitoring.{dataset}-{namespace}`. For example: +`metrics-elasticsearch.stack_monitoring.shard-default`. + +The settings and mappings for these data streams are determined by an index +template named `metrics-{product}.stack_monitoring.{dataset}`. For example: +`metrics-elasticsearch.stack_monitoring.shard`. + +To change the settings of each data stream, edit the +`metrics-{product}.stack_monitoring.{dataset}@custom` component template that +already exists. You can do this in {kib}: + +* Navigate to *Stack Management* > *Index Management* > *Component Templates*. +* Search for the component template. +* Select the *Edit* action. + +You can also use the {es} API: + +* Retrieve the component template using the <>. +* Edit the component template. +* Store the updated component template using the <>. + +After changing the component template, the updated settings are only applied +to the data stream's new backing indices. +<> to immediately +apply the updated settings to the data stream’s write index. + +[[config-monitoring-data-streams-metricbeat-8]] +=== Configuring data streams created by {metricbeat} 8 + +When <>, data is stored +in a set of data streams called `.monitoring-{product}-8-mb`. For example: +`.monitoring-es-8-mb`. + +The settings and mappings for these data streams are determined by an index +template named `.monitoring-{product}-mb`. For example: `.monitoring-es-mb`. You +can alter the settings of each data stream by cloning this index template and +editing it. + +WARNING: You need to repeat this procedure when upgrading the {stack} to get the +latest updates to the default monitoring index templates. -[float] -=== Configuring indices created by {metricbeat} +You can clone index templates in {kib}: -<> are used to configure the indices that -store the monitoring data that {metricbeat} collects from a cluster. +* Navigate to *Stack Management* > *Index Management* > *Index Templates*. +* From the *View* dropdown, select *System templates*. +* Search for the index template. +* Select the *Clone* action. +* Change the name, for example into `custom_monitoring`. +* Set the priority to `500`, to ensure it overrides the default index template. +* Specify the settings you want to change in the `settings` section. +* Save the cloned template. -You can retrieve the templates through the `_template` API: +You can also use the {es} API: + +* Retrieve the index template using the <>. +* Edit the index template: set the template `priority` to `500`, and specify the +settings you want to change in the `settings` section. +* Store the updated index template under a different name, for example +`custom_monitoring`, using the +<>. + +NOTE: {metricbeat} 8 uses <>, rather than +legacy templates. + +After changing the index template, the updated settings are only applied to the +data stream's new backing indices. +<> to immediately +apply the updated settings to the data stream’s write index. + +[[config-monitoring-indices-metricbeat-7-internal-collection]] +=== Configuring indices created by {metricbeat} 7 or internal collection + +When monitoring <> or +{filebeat-ref}/monitoring-internal-collection.html[internal collection], data is +stored in a set of indices called either: + +* `.monitoring-{product}-7-mb-{date}`, when using {metricbeat} 7. +* `.monitoring-{product}-7-{date}`, when using internal collection. + +The settings and mappings for these indices are determined by +<> named `.monitoring-{product}`. +You can retrieve these templates in {kib} by navigating to *Stack Management* > +*Index Management* > *Index Templates*, or by using the {es} `_template` API: [source,console] ----------------------------------- -GET /_index_template/.monitoring-es-mb* ----------------------------------- +---- +GET /_template/.monitoring-* +---- -By default, the template configures one shard and one replica for the -monitoring indices. To override the default settings, add your own template: +To change the settings of the indices, add a custom index template. You can do +that in {kib}, or using the {es} API: -. Set `index_patterns` to match existing `.monitoring-{product}-8-*` indices. -. Set the template `priority` to `1`. This ensures your template is -applied after the default template, which has an priority of 0. -. Specify the `number_of_shards` and/or `number_of_replicas` in the `settings` +* Set `index_patterns` to match the `.monitoring-{product}-7-*` indices. +* Set the template `order` to `1`. This ensures your template is +applied after the default template, which has an order of 0. +* Specify the `number_of_shards` and/or `number_of_replicas` in the `settings` section. -For example, the following template increases the number of shards to five -and the number of replicas to two. - [source,console] ----------------------------------- -PUT /_index_template/custom_monitoring +---- +PUT /_template/custom_monitoring { - "index_patterns": [ - ".monitoring-beats-8-*", - ".monitoring-es-8-*", - ".monitoring-kibana-8-*", - ".monitoring-logstash-8-*", - ".monitoring-ent-search-8-*" - ], - "priority": 1, - "template": { - "settings": { - "number_of_shards": 5, - "number_of_replicas": 2 - } - } + "index_patterns": [".monitoring-beats-7-*", ".monitoring-es-7-*", ".monitoring-kibana-7-*", ".monitoring-logstash-7-*"], + "order": 1, + "settings": { + "number_of_shards": 5, + "number_of_replicas": 2 + } } ----------------------------------- +---- -////////////////////////// +After changing the index template, the updated settings are only applied to new +indices. +//// [source,console] --------------------------------------------------- -DELETE /_index_template/custom_monitoring --------------------------------------------------- +---- +DELETE /_template/custom_monitoring +---- // TEST[continued] - -////////////////////////// - -IMPORTANT: Only set the `number_of_shards` and `number_of_replicas` in the -settings section. Overriding other monitoring template settings could cause -your monitoring dashboards to stop working correctly. +//// \ No newline at end of file