Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DOCS] Reorg monitoring configuration for re-use #42547

Merged
merged 4 commits into from
May 28, 2019
Merged

[DOCS] Reorg monitoring configuration for re-use #42547

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
800764c
[DOCS] Reorg monitoring configuration for re-use
lcawl May 24, 2019
b6db673
[DOCS] Fixes ordered list
lcawl May 24, 2019
773f91c
[DOCS] Fixes tagged region
lcawl May 24, 2019
ce854ff
[DOCS] Updates description of example
lcawl May 27, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 68 additions & 53 deletions docs/reference/monitoring/configuring-metricbeat.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ image::monitoring/images/metricbeat.png[Example monitoring architecture]
To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].

//NOTE: The tagged regions are re-used in the Stack Overview.

. Enable the collection of monitoring data. Set
`xpack.monitoring.collection.enabled` to `true` on each node in the production
cluster. By default, it is is disabled (`false`).
Expand Down Expand Up @@ -71,13 +73,13 @@ PUT _cluster/settings
Leave `xpack.monitoring.enabled` set to its default value (`true`).
--

. On each {es} node in the production cluster:

.. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}].
. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}] on each
{es} node in the production cluster.

.. Enable the {es} module in {metricbeat}. +
. Enable the {es} {xpack} module in {metricbeat} on each {es} node. +
+
--
// tag::enable-es-module[]
For example, to enable the default configuration in the `modules.d` directory,
run the following command:

Expand All @@ -89,39 +91,57 @@ metricbeat modules enable elasticsearch-xpack
For more information, see
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and
{metricbeat-ref}/metricbeat-module-elasticsearch.html[{es} module].
--

.. By default the module will collect {es} monitoring metrics from `http://localhost:9200`.
If the local {es} node has a different address, you must specify it via the `hosts` setting
in the `modules.d/elasticsearch-xpack.yml` file.

.. If Elastic {security-features} are enabled, you must also provide a user ID
and password so that {metricbeat} can collect metrics successfully.

... Create a user on the production cluster that has the
{stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role].
Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
// end::enable-es-module[]
--

... Add the `username` and `password` settings to the {es} module configuration
file.
. Configure the {es} {xpack} module in {metricbeat} on each {es} node. +
+
--
For example, add the following settings in the `modules.d/elasticsearch-xpack.yml` file:
// tag::configure-es-module[]
The `modules.d/elasticsearch-xpack.yml` file contains the following settings:

[source,yaml]
----------------------------------
- module: elasticsearch
...
username: remote_monitoring_user
password: YOUR_PASSWORD
- module: elasticsearch
metricsets:
- ccr
- cluster_stats
- index
- index_recovery
- index_summary
- ml_job
- node_stats
- shard
period: 10s
hosts: ["http://localhost:9200"]
#username: "user"
#password: "secret"
xpack.enabled: true
----------------------------------
--

.. If you configured {es} to use <<configuring-tls,encrypted communications>>,
you must access it via HTTPS. For example, use a `hosts` setting like
`https://localhost:9200` in the `modules.d/elasticsearch-xpack.yml` file.
By default, the module collects {es} monitoring metrics from
`http://localhost:9200`. If that host and port number are not correct, you must
update the `hosts` setting. If you configured {es} to use encrypted
communications, you must access it via HTTPS. For example, use a `hosts` setting
like `https://localhost:9200`.
// end::configure-es-module[]

.. Identify where to send the monitoring data. +
// tag::remote-monitoring-user[]
If Elastic {security-features} are enabled, you must also provide a user ID
and password so that {metricbeat} can collect metrics successfully:

.. Create a user on the production cluster that has the
{stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role].
Alternatively, use the
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].

.. Add the `username` and `password` settings to the {es} module configuration
file.
// end::remote-monitoring-user[]
--

. Identify where to send the monitoring data. +
+
--
TIP: In production environments, we strongly recommend using a separate cluster
Expand All @@ -136,48 +156,43 @@ configuration file (`metricbeat.yml`):
[source,yaml]
----------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>

# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
----------------------------------
<1> In this example, the data is stored on a monitoring cluster with nodes
`es-mon-1` and `es-mon-2`.

If you configured the monitoring cluster to use encrypted communications, you
must access it via HTTPS. For example, use a `hosts` setting like
`https://es-mon-1:9200`.

IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
cluster that stores the monitoring data must have at least one
<<ingest,ingest node>>.

For more information about these configuration options, see
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

.. If {es} {security-features} are enabled on the monitoring cluster, you
must provide a valid user ID and password so that {metricbeat} can send metrics
successfully.
If {es} {security-features} are enabled on the monitoring cluster, you must
provide a valid user ID and password so that {metricbeat} can send metrics
successfully:

... Create a user on the monitoring cluster that has the
.. Create a user on the monitoring cluster that has the
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
Alternatively, use the
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].

... Add the `username` and `password` settings to the {es} output information in
the {metricbeat} configuration file (`metricbeat.yml`):
+
--
[source,yaml]
----------------------------------
output.elasticsearch:
...
username: remote_monitoring_user
password: YOUR_PASSWORD
----------------------------------
--
.. Add the `username` and `password` settings to the {es} output information in
the {metricbeat} configuration file.

.. If you configured the monitoring cluster to use
<<configuring-tls,encrypted communications>>, you must access it via
HTTPS. For example, use a `hosts` setting like `https://es-mon-1:9200` in the
`metricbeat.yml` file.
For more information about these configuration options, see
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

. <<starting-elasticsearch,Start {es}>>.
. <<starting-elasticsearch,Start {es}>> on each node.

. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}].
. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}] on each node.

. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}].