[DOCS] Clarify configuring indices/data streams used by stack monitoring

docs/reference/monitoring/indices.asciidoc

@@ -1,79 +1,147 @@
 [role="xpack"]
 [[config-monitoring-indices]]
-== Configuring indices for monitoring
+== Configuring data streams/indices for monitoring
 
-Sometimes the default index settings might not work for your situation. For
-example, you might want to change index lifecycle management (ILM) settings,
-add custom mappings, or set the number of shards and replicas
-for the monitoring indices.
+Monitoring data is stored in data streams or indices in {es}. The default data
+stream or index settings may not work for your situation. For example, you might
+want to change index lifecycle management (ILM) settings, add custom mappings,
+or change the number of shards and replicas. The steps to change these
+settings depend on the monitoring method:
 
-You can change the default behavior. The steps you follow depend on which data
-collection method is used.
+* <<config-monitoring-data-streams-elastic-agent>>
+* <<config-monitoring-data-streams-metricbeat-8>>
+* <<config-monitoring-indices-metricbeat-7-internal-collection>> (the default 
+for the {ess} on {ecloud})
 
-[float]
+IMPORTANT: Changing mappings or settings can cause your monitoring dashboards to
+stop working correctly.
+
+[[config-monitoring-data-streams-elastic-agent]]
 === Configuring data streams created by {agent}
 
-{agent} uses data streams to store time series data across multiple indices
-while giving you a single named resource for requests. You can alter the
-settings of each dataset by configuring an `@custom` component template.
-For more information, refer to {fleet-guide}/data-streams.html[Data streams].
+When <<configuring-elastic-agent,monitoring using {agent}>>, data is stored in a
+set of data streams named
+`metrics-{product}.stack_monitoring.{dataset}-{namespace}`. For example:
+`metrics-elasticsearch.stack_monitoring.shard-default`. 
+
+The settings and mappings for these data streams are determined by an index
+template named `metrics-{product}.stack_monitoring.{dataset}`. For example:
+`metrics-elasticsearch.stack_monitoring.shard`. 
+
+To change the settings of each data stream, edit the
+`metrics-{product}.stack_monitoring.{dataset}@custom` component template that
+already exists. You can do this in {kib}:
+
+* Navigate to *Stack Management* > *Index Management* > *Component Templates*.
+* Search for the component template.
+* Select the *Edit* action.
+
+You can also use the {es} API:
+
+* Retrieve the component template using the <<getting-component-templates,get 
+component template API>>.
+* Edit the component template.
+* Store the updated component template using the <<indices-component-template,
+update component template API>>.
+
+After changing the component template, the updated settings are only applied
+to the data stream's new backing indices.
+<<manually-roll-over-a-data-stream,Roll over the data stream>> to immediately
+apply the updated settings to the data stream’s write index.
+
+[[config-monitoring-data-streams-metricbeat-8]]
+=== Configuring data streams created by {metricbeat} 8
+
+When <<configuring-metricbeat,monitoring using {metricbeat} 8>>, data is stored
+in a set of data streams called `.monitoring-{product}-8-mb`. For example:
+`.monitoring-es-8-mb`. 
+
+The settings and mappings for these data streams are determined by an index
+template named `.monitoring-{product}-mb`. For example: `.monitoring-es-mb`. You
+can alter the settings of each data stream by cloning this index template and
+editing it. 
+
+WARNING: You need to repeat this procedure when upgrading the {stack} to get the
+latest updates to the default monitoring index templates.
 
-[float]
-=== Configuring indices created by {metricbeat}
+You can clone index templates in {kib}:
 
-<<indices-templates-v1,Index templates>> are used to configure the indices that
-store the monitoring data that {metricbeat} collects from a cluster.
+* Navigate to *Stack Management* > *Index Management* > *Index Templates*.
+* From the *View* dropdown, select *System templates*.
+* Search for the index template.
+* Select the *Clone* action.
+* Change the name, for example into `custom_monitoring`.
+* Set the priority to `500`, to ensure it overrides the default index template.
+* Specify the settings you want to change in the `settings` section.
+* Save the cloned template.
 
-You can retrieve the templates through the `_template` API:
+You can also use the {es} API:
+
+* Retrieve the index template using the <<indices-get-template,get index 
+template API>>.
+* Edit the index template: set the template `priority` to `500`, and specify the 
+settings you want to change in the `settings` section.
+* Store the updated index template under a different name, for example 
+`custom_monitoring`, using the 
+<<indices-put-template,create index template API>>.
+
+NOTE: {metricbeat} 8 uses <<index-templates,composable templates>>, rather than
+legacy templates.
+
+After changing the index template, the updated settings are only applied to the
+data stream's new backing indices.
+<<manually-roll-over-a-data-stream,Roll over the data stream>> to immediately
+apply the updated settings to the data stream’s write index.
+
+[[config-monitoring-indices-metricbeat-7-internal-collection]]
+=== Configuring indices created by {metricbeat} 7 or internal collection
+
+When monitoring <<configuring-metricbeat,using {metricbeat} 7>> or
+{filebeat-ref}/monitoring-internal-collection.html[internal collection], data is
+stored in a set of indices called either:
+
+* `.monitoring-{product}-7-mb-{date}`, when using {metricbeat} 7.
+* `.monitoring-{product}-7-{date}`, when using internal collection.
+
+The settings and mappings for these indices are determined by
+<<indices-templates-v1,legacy index templates>> named `.monitoring-{product}`.
+You can retrieve these templates in {kib} by navigating to *Stack Management* >
+*Index Management* > *Index Templates*, or by using the {es} `_template` API:
 
 [source,console]
-----------------------------------
-GET /_index_template/.monitoring-es-mb*
-----------------------------------
+----
+GET /_template/.monitoring-*
+----
 
-By default, the template configures one shard and one replica for the
-monitoring indices. To override the default settings, add your own template:
+To change the settings of the indices, add a custom index template. You can do 
+that in {kib}, or using the {es} API:
 
-. Set `index_patterns` to match existing `.monitoring-{product}-8-*` indices.
-. Set the template `priority` to `1`. This ensures your template is
-applied after the default template, which has an priority of 0.
-. Specify the `number_of_shards` and/or `number_of_replicas` in the `settings`
+* Set `index_patterns` to match the `.monitoring-{product}-7-*` indices.
+* Set the template `order` to `1`. This ensures your template is
+applied after the default template, which has an order of 0.
+* Specify the `number_of_shards` and/or `number_of_replicas` in the `settings`
 section.
 
-For example, the following template increases the number of shards to five
-and the number of replicas to two.
-
 [source,console]
-----------------------------------
-PUT /_index_template/custom_monitoring
+----
+PUT /_template/custom_monitoring
 {
-  "index_patterns": [
-  ".monitoring-beats-8-*",
-  ".monitoring-es-8-*",
-  ".monitoring-kibana-8-*",
-  ".monitoring-logstash-8-*",
-  ".monitoring-ent-search-8-*"
-  ],
-  "priority": 1,
-  "template": {
-    "settings": {
-      "number_of_shards": 5,
-      "number_of_replicas": 2
-    }
-  } 
+  "index_patterns": [".monitoring-beats-7-*", ".monitoring-es-7-*", ".monitoring-kibana-7-*", ".monitoring-logstash-7-*"],
+  "order": 1,
+  "settings": {
+    "number_of_shards": 5,
+    "number_of_replicas": 2
+  }
 }
-----------------------------------
+----
 
-//////////////////////////
+After changing the index template, the updated settings are only applied to new
+indices.
 
+////
 [source,console]
---------------------------------------------------
-DELETE /_index_template/custom_monitoring
---------------------------------------------------
+----
+DELETE /_template/custom_monitoring
+----
 // TEST[continued]
-
-//////////////////////////
-
-IMPORTANT: Only set the `number_of_shards` and `number_of_replicas` in the
-settings section. Overriding other monitoring template settings could cause
-your monitoring dashboards to stop working correctly.
+////