Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions doc/monitoring/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -194,3 +194,120 @@ via configuration.
format: 'json'

.. _grafana-dashboard:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to move it to it's own page. Separate page is bit more "googlable"

-------------------------------------------------------------------------------
Grafana dashboard
-------------------------------------------------------------------------------

Tarantool Grafana dashboard is available as part of
`Grafana Official & community built dashboards <https://grafana.com/grafana/dashboards>`_.
You can find version for Prometheus datasource on
`this page <https://grafana.com/grafana/dashboards/13054>`_ and version for
InfluxDB datasource on `this page <https://grafana.com/grafana/dashboards/12567>`_.
Tarantool Grafana dashboard is a ready for import template with basic memory, space operations
and HTTP load panels, based on default ``metrics`` package functionality.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to add reference to metrics docs or github repo


Dashboards require using ``metrics`` **0.5.0** or newer;
``'alias'`` :ref:`global label <tarantool-metrics>` must be set on each instance
to properly display panels (e.g. provided with ``cartridge.roles.metrics`` role).

**Usage**

#. Set up your monitoring stack. Since there are `Prometheus <https://prometheus.io/>`_
and `InfluxDB <https://www.influxdata.com/>`_ datasource Grafana dashboards,
you can use

- `Telegraf <https://www.influxdata.com/time-series-platform/telegraf/>`_
as a server agent for collecting metrics, `InfluxDB <https://www.influxdata.com/>`_
as a time series database for storing metrics, `Grafana <https://grafana.com/>`_
as a visualization platform; or
- `Prometheus <https://prometheus.io/>`_ as both server agent for collecting metrics
and time series database for storing metrics, `Grafana <https://grafana.com/>`_
as a visualization platform.

For issues concerning set up of Prometheus, Telegraf, InfluxDB or Grafana instances
please refer to corresponding project's documentation.

#. Configure your Tarantool instances to output metrics with proper format
and configure corresponding server agent to collect them.

To collect metrics for Prometheus, first off you must set up metrics output with
``prometheus`` format. You can use :ref:`cartridge.roles.metrics <cartridge-role>`
configuration or set up :ref:`output plugin <prometheus>` manually.
To start collecting metrics, add a `job <https://prometheus.io/docs/prometheus/latest/getting_started/#configure-prometheus-to-monitor-the-sample-targets>`_
to Prometheus configuration with each Tarantool instance URI as a target and
metrics path as it was configured on Tarantool instances:

.. code-block:: yaml

scrape_configs:
- job_name: "example_project"
static_configs:
- targets:
- "example_project:8081"
- "example_project:8082"
- "example_project:8083"
metrics_path: "/metrics/prometheus"

To collect metrics for InfluxDB, you must use Telegraf agent.
First off, configure Tarantool metrics output in ``json`` format
with :ref:`cartridge.roles.metrics <cartridge-role>` configuration or
corresponding :ref:`output plugin <json>`. To start collecting metrics,
add `http input <https://github.com/influxdata/telegraf/blob/release-1.17/plugins/inputs/http/README.md>`_
to Telegraf configuration including each Tarantool instance metrics URL:

.. code-block:: text

[[inputs.http]]
urls = [
"http://example_project:8081/metrics/json",
"http://example_project:8082/metrics/json",
"http://example_project:8083/metrics/json"
]
timeout = "30s"
tag_keys = [
"metric_name",
"label_pairs_alias",
"label_pairs_quantile",
"label_pairs_path",
"label_pairs_method",
"label_pairs_status",
"label_pairs_operation"
]
insecure_skip_verify = true
interval = "10s"
data_format = "json"
name_prefix = "example_project_"
fieldpass = ["value"]

Be sure to include each label key as ``label_pairs_<key>`` so it will be
extracted with plugin. For example, if you use :code:`{ state = 'ready' }` labels
somewhere in metric collectors, add ``label_pairs_state`` tag key.

If you connect Telegraf instance to InfluxDB storage, metrics will be stored
with ``"<name_prefix>http"`` measurement (``"example_project_http"`` in our example).

#. Import a dashboard from `Grafana Official & community built dashboards <https://grafana.com/grafana/dashboards>`_.

Open Grafana import menu.

.. image:: images/grafana-import-v6.png
:align: center

To import specific dashboard, choose one of the following options:

- paste dashboard id (``12567`` for InfluxDB dashboard, ``13054`` for Prometheus dashboard), or
- paste link to dashboard (https://grafana.com/grafana/dashboards/12567 for InfluxDB dashboard,
https://grafana.com/grafana/dashboards/13054 for Prometheus dashboard), or
- paste dashboard json file contents, or
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to provide direct link to download json file?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be possible, according to the Sphinx docs (:download: role), but we never tried

Copy link
Member Author

@DifferentialOrange DifferentialOrange Feb 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible (e.g. here https://github.com/tarantool/grafana-dashboard/releases/tag/0.2.2) to make a link to specific revision, but we will need to update it every time I upload a new version.

- upload dashboard json file.

Set dashboard name, folder, uid (if needed), and datasource-related query parameters
(InfluxDB source, measurement and policy or Prometheus source, job and rate time range).

.. image:: images/grafana-import-setup-v6.png
:align: center

If no data present on graphs, ensure that you set up datasource and job/measurement correctly.
If no data present on rps graphs on Prometheus table, ensure that
your rate time range parameter is at least twice as Prometheus scrape interval.
Binary file added doc/monitoring/images/grafana-import-setup-v6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/monitoring/images/grafana-import-v6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.