Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Update Kibana monitoring tasks #23736

Merged
merged 7 commits into from
Oct 4, 2018
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ include::introduction.asciidoc[]

include::setup.asciidoc[]

include::monitoring/monitoring-xkib.asciidoc[]
include::monitoring/monitoring-kibana.asciidoc[]

include::security/securing-kibana.asciidoc[]

Expand Down Expand Up @@ -76,4 +76,6 @@ include::release-notes/highlights.asciidoc[]

include::migration.asciidoc[]

include::CHANGELOG.asciidoc[]
include::CHANGELOG.asciidoc[]

include::redirects.asciidoc[]
4 changes: 0 additions & 4 deletions docs/monitoring/configuring-monitoring.asciidoc

This file was deleted.

4 changes: 0 additions & 4 deletions docs/monitoring/getting-started.asciidoc

This file was deleted.

2 changes: 1 addition & 1 deletion docs/monitoring/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ metrics. Each node and instance is considered unique based on its persistent
UUID, which is written to the <<settings,`path.data`>> directory when the node
or instance starts.

* <<monitoring-xpack-kibana>>
* <<monitoring-kibana>>
* <<monitoring-data>>

--
Expand Down
4 changes: 0 additions & 4 deletions docs/monitoring/monitoring-elasticsearch.asciidoc

This file was deleted.

101 changes: 98 additions & 3 deletions docs/monitoring/monitoring-kibana.asciidoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,100 @@
[role="xpack"]
[[monitoring-kibana]]
== Monitoring Kibana
== Configuring monitoring in {kib}
++++
<titleabbrev>Configuring monitoring</titleabbrev>
++++

See
{kibana-ref}/monitoring-xpack-kibana.html[Monitoring {kib}].
If you enable the collection of monitoring data in your cluster, you can
optionally collect metrics about {kib}.

The following method involves sending the metrics to {es}, which ultimately
routes them to the monitoring cluster. For an alternative method, see
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following method involves sending the metrics to Elasticsearch, which ultimately routes them to the monitoring cluster.

"Elasticsearch" and "monitoring cluster" are both Elasticsearch clusters. So I wonder if this wording might be confusing.

In our internal discussions we often use the terms "production cluster" and "monitoring cluster" to draw the distinction between the two Elasticsearch clusters in the context of Monitoring. And, of course, users might use the same cluster for both purposes, meaning that their production and monitoring clusters are one and the same.

I'm not sure if this needs rewording but I thought I'd bring this up. Ultimately I trust your judgement on what best to do here 😄.

<<monitoring-metricbeat>>.

To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].

. Enable the collection of monitoring data in the {stack}. Set
`xpack.monitoring.collection.enabled` to `true` on the production cluster. +
+
--
For example, you can use the following APIs to review and change this setting:

[source,js]
----------------------------------
GET _cluster/settings

PUT _cluster/settings
{
"persistent": {
"xpack.monitoring.collection.enabled": true
}
}
----------------------------------

For more information, see {ref}/monitoring-settings.html[Monitoring settings in {es}]
and {ref}/cluster-update-settings.html[Cluster update settings].
--

. Verify that `xpack.monitoring.enabled` and
`xpack.monitoring.kibana.collection.enabled` are set to `true` in the
`kibana.yml` file on your production cluster. These are the default values. For
more information, see <<monitoring-settings-kb,Monitoring settings in {kib}>>.

. Identify where to send monitoring data. {kib} automatically
sends metrics to the {es} cluster specified in the `elasticsearch.url` setting
in the `kibana.yml` file. This property has a default value of
`http://localhost:9200`. +
+
--
[TIP]
===============================
In production environments, we strongly recommend using a separate cluster
(referred to as the _monitoring cluster_) to store the data. Using a separate
monitoring cluster prevents production cluster outages from impacting your
ability to access your monitoring data. It also prevents monitoring activities
from impacting the performance of your production cluster.

If {security} is enabled on the production cluster, use an HTTPS URL such
as `https://<your_production_cluster>:9200` in this setting.
===============================

--

. If {security} is enabled on the production cluster:

.. Verify that there is a
valid user ID and password in the `elasticsearch.username` and
`elasticsearch.password` settings in the `kibana.yml` file. These values are
used when {kib} sends monitoring data to the production cluster.

.. Configure {kib} to encrypt communications between the {kib} server and the
production cluster. This set up involves generating a server certificate and
setting `server.ssl.*` and `elasticsearch.ssl.certificateAuthorities` settings
in the `kibana.yml` file on the {kib} server. For example:
+
--
[source,yaml]
--------------------------------------------------------------------------------
server.ssl.key: /path/to/your/server.key
server.ssl.certificate: /path/to/your/server.crt
--------------------------------------------------------------------------------

If you are using your own certificate authority to sign certificates, specify
the location of the PEM file in the `kibana.yml` file:

[source,yaml]
--------------------------------------------------------------------------------
elasticsearch.ssl.certificateAuthorities: /path/to/your/cacert.pem
--------------------------------------------------------------------------------

For more information, see <<using-kibana-with-security>>.
--

. Restart {kib}.

. <<monitoring-data,View the monitoring data in {kib}>>.

include::monitoring-metricbeat.asciidoc[]
include::{kib-repo-dir}/settings/monitoring-settings.asciidoc[]
5 changes: 0 additions & 5 deletions docs/monitoring/monitoring-logstash.asciidoc

This file was deleted.

117 changes: 117 additions & 0 deletions docs/monitoring/monitoring-metricbeat.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
[role="xpack"]
[[monitoring-metricbeat]]
=== Monitoring {kib} with {metricbeat}

beta[] In 6.4 and later, you can use {metricbeat} to collect and ship data about
{kib}, rather than routing it through {es} as described in <<monitoring-kibana>>.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Configuring Monitoring in Kibana" doc which lists the old/current way of monitoring has this line in it:

To learn about monitoring in general, see Monitoring the Elastic Stack.

Do you think we should add this line in this doc as well, perhaps for users who come to this doc without reading the other one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed! I've made this change (and several others) and refreshed the preview server.

. Enable the collection of monitoring data in the {stack}. Set
`xpack.monitoring.collection.enabled` to `true` on the production cluster. +
+
--
For example, you can use the following APIs to review and change this setting:

[source,js]
----------------------------------
GET _cluster/settings

PUT _cluster/settings
{
"persistent": {
"xpack.monitoring.collection.enabled": true
}
}
----------------------------------

For more information, see {ref}/monitoring-settings.html[Monitoring settings in {es}]
and {ref}/cluster-update-settings.html[Cluster update settings].
--

. Disable the default collection of {kib} monitoring metrics. +
+
--
Add the following setting in the `kibana.yml` file on your production cluster:

[source,yaml]
----------------------------------
xpack.monitoring.kibana.collection.enabled: false
----------------------------------

For more information, see
<<monitoring-settings-kb,Monitoring settings in {kib}>>.
//TBD: What does xpack.monitoring.enabled need to be set to?
--

. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}] on the
same node as {kib}.

. Enable the {kib} module in {metricbeat}. +
+
--
For example, to enable the default configuration in the `modules.d` directory,
run the following command:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
metricbeat modules enable kibana
----------------------------------------------------------------------

Alternatively, you can enable module configs in the `metricbeat.yml` file. For
more information, see
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and
{metricbeat-ref}/metricbeat-module-kibana.html[{kib} module].
--

. Configure the {kib} module in {metricbeat}. +
+
--
For example, you can change the default module configuration by modifying the
Copy link
Contributor

@ycombinator ycombinator Oct 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the wording ought to be a bit more prescriptive here. In order for things to work correctly with metricbeat monitoring kibana, users need to have the exact configuration in modules.d/kibana.yml as shown in the code snippet below (except for the hosts setting, of course, as that might depend on each user's setup).

`kibana.yml` file in the `modules.d` directory:

[source,yaml]
----------------------------------
- module: kibana
metricsets:
- stats
period: 10s
hosts: ["http://localhost:5601"] <1>
xpack.enabled: true
----------------------------------
<1> This setting identifies the host and port number that are used to access {kib}.

NOTE: If you configured {kib} to use <<configuring-tls,encrypted communications>>,
you must access it via HTTPS. For example, `https://localhost:5601`.

--

. Identify where to send the monitoring data. +
+
--
TIP: In production environments, we strongly recommend using a separate cluster
(referred to as the _monitoring cluster_) to store the data. Using a separate
monitoring cluster prevents production cluster outages from impacting your
ability to access your monitoring data. It also prevents monitoring activities
from impacting the performance of your production cluster.

For example, specify the {es} output information in the `kibana.yml` file in the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this should be ... the {es} output information in the metricbeat.yml file:

`modules.d` directory:

[source,yaml]
----------------------------------
output.elasticsearch:
hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>
----------------------------------
<1> In this example, the data is stored on a monitoring cluster with nodes
`es-mon-1` and `es-mon-2`.

NOTE: If you configured the monitoring cluster to use
{ref}/configuring-tls.html[encrypted communications], you must access it via
HTTPS. For example, `https://es-mon-1:9200`.

--

. Start {kib}.

. Start {metricbeat}.

. <<monitoring-data,View the monitoring data in {kib}>>.
Loading