Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Config] Update 'Flight recorder configuration' to using a new config #3995

Merged
merged 8 commits into from
Jan 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
flightrec:
enabled: true
logs_size: 10485800
logs_log_level: 5
metrics_period: 240
metrics_interval: 0.5
requests_size: 10485780

groups:
group001:
replicasets:
replicaset001:
instances:
instance001:
iproto:
listen:
- uri: '127.0.0.1:3301'
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
instance001:
212 changes: 34 additions & 178 deletions doc/enterprise/flight_recorder.rst
Original file line number Diff line number Diff line change
@@ -1,218 +1,74 @@
.. _enterprise-flight-recorder:
.. _enterprise-flight-recorder:

Flight recorder
===============

**Example on GitHub**: `flightrec <https://github.com/tarantool/doc/tree/latest/doc/code_snippets/snippets/config/instances.enabled/flightrec>`_

The flight recorder available in the Enterprise Edition is an event collection tool that
gathers various information about a working Tarantool instance, such as:

* logs

* metrics

* requests and responses

This information helps you investigate incidents related
to :ref:`crashing <admin-disaster_recovery>` a Tarantool instance.


.. _enable:
.. _enterprise-flight-recorder_enable:

Enabling the flight recorder
----------------------------

The flight recorder is disabled by default and can be enabled and configured for
a specific Tarantool instance.
To enable the flight recorder, set the ``flightrec_enabled``
:doc:`configuration option </reference/configuration/index>` to ``true``.
This option is :ref:`dynamic <box_cfg_params>` and can be changed at runtime by calling ``box.cfg{}``:
To enable the flight recorder, set the :ref:`flightrec.enabled <configuration_reference_flightrec_enabled>`
configuration option to ``true``.

.. literalinclude:: /code_snippets/snippets/config/instances.enabled/flightrec/config.yaml
:language: yaml
:start-at: flightrec:
:end-at: enabled: true
:dedent:

.. code-block:: lua

box.cfg{ flightrec_enabled = true }
After ``flightrec.enabled`` is set to ``true``, the flight recorder starts collecting data in the flight recording file ``current.ttfr``.
This file is stored in the ``snapshot.dir`` directory.
By default, the directory is ``var/lib/{{ instance_name }}/<file_name>.ttfr``.

After ``flightrec_enabled`` is set to ``true``, the flight recorder starts collecting data in the flight recording file ``current.ttfr``.
This file is stored in the :ref:`memtx_dir <cfg_basic-memtx_dir>` directory.
If the instance crashes and reboots, Tarantool rotates the flight recording:
``current.ttfr`` is renamed to ``<timestamp>.ttfr`` (for example, ``20230411T050721.ttfr``)
and the new ``current.ttfr`` file is created for collecting data.
In the case of correct shutdown (for example, using ``os.exit()``),
Tarantool continues writing to the existing ``current.ttfr`` file after restart.

.. NOTE::
.. NOTE::

Note that old flight recordings should be removed manually.


.. _enterprise-flight-recorder_configure:

.. _config:

Configuration
-------------

This section describes options related to the flight recorder configuration.
Note that all options are :ref:`dynamic <box_cfg_params>` and can be changed at runtime.

.. TODO not implemented yet
.. _config-directory:

Flight recording directory
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. _cfg_flightrec_dir:

.. confval:: flightrec_dir

Specifies the directory used to store flight recordings (``*.ttfr`` files).

| Type: string
| Default: :ref:`memtx_dir <cfg_basic-memtx_dir>`
| Environment variable: TT_FLIGHTREC_DIR



.. _config-logs:

Logs
~~~~

This section describes the flight recorder settings related to :ref:`logging <cfg_logging-log>`.
For example, you can configure a more detailed logging level in the flight recorder for deeper analysis.

* :ref:`flightrec_logs_size <cfg_flightrec_logs_size>`
* :ref:`flightrec_logs_max_msg_size <cfg_flightrec_logs_max_msg_size>`
* :ref:`flightrec_logs_log_level <cfg_flightrec_logs_log_level>`

.. _cfg_flightrec_logs_size:

.. confval:: flightrec_logs_size

Specifies the size (in bytes) of the log storage.
You can set this option to ``0`` to disable the log storage.

| Type: integer
| Default: 10485760
| Environment variable: TT_FLIGHTREC_LOGS_SIZE


.. _cfg_flightrec_logs_max_msg_size:

.. confval:: flightrec_logs_max_msg_size

Specifies the maximum size (in bytes) of the log message.
The log message is truncated if its size exceeds this limit.

| Type: integer
| Default: 4096
| Maximum: 16384
| Environment variable: TT_FLIGHTREC_LOGS_MAX_MSG_SIZE


.. _cfg_flightrec_logs_log_level:

.. confval:: flightrec_logs_log_level

Specifies the level of detail the log has.
You can learn more about log levels from the :ref:`log_level <cfg_logging-log_level>`
option description.
Note that the ``flightrec_logs_log_level`` value might differ from ``log_level``.

| Type: integer
| Default: 6
| Environment variable: TT_FLIGHTREC_LOGS_LOG_LEVEL


.. _config-metrics:

Metrics
~~~~~~~

This section describes the flight recorder settings related to collecting
:ref:`metrics <metrics-reference>`.

* :ref:`flightrec_metrics_period <cfg_flightrec_metrics_period>`
* :ref:`flightrec_metrics_interval <cfg_flightrec_metrics_interval>`

.. _cfg_flightrec_metrics_period:

.. confval:: flightrec_metrics_period

Specifies the time period (in seconds) that defines how long metrics are stored from the moment of dump.
So, this value defines how much historical metrics data is collected up to the moment of crash.
The frequency of metric dumps is defined by :ref:`flightrec_metrics_interval <cfg_flightrec_metrics_interval>`.

| Type: integer
| Default: 180
| Environment variable: TT_FLIGHTREC_METRICS_PERIOD


.. _cfg_flightrec_metrics_interval:

.. confval:: flightrec_metrics_interval

Specifies the time interval (in seconds) that defines the frequency of dumping metrics.
This value shouldn't exceed :ref:`flightrec_metrics_period <cfg_flightrec_metrics_period>`.

| Type: number
| Default: 1.0
| Minimum: 0.001
| Environment variable: TT_FLIGHTREC_METRICS_INTERVAL

.. NOTE::

Given that the average size of a metrics entry is 2 kB,
you can estimate the size of the metrics storage as follows:

.. code-block:: console

(flightrec_metrics_period / flightrec_metrics_interval) * 2 kB



.. _config-requests:

Requests
~~~~~~~~

This section lists the flight recorder settings related to
storing the :ref:`request and response <internals-requests_responses>` data.

* :ref:`flightrec_requests_size <cfg_flightrec_requests_size>`
* :ref:`flightrec_requests_max_req_size <cfg_flightrec_requests_max_req_size>`
* :ref:`flightrec_requests_max_res_size <cfg_flightrec_requests_max_res_size>`

.. _cfg_flightrec_requests_size:

.. confval:: flightrec_requests_size

Specifies the size (in bytes) of storage for the request and response data.
You can set this parameter to ``0`` to disable a storage of requests and responses.

| Type: integer
| Default: 10485760
| Environment variable: TT_FLIGHTREC_REQUESTS_SIZE



.. _cfg_flightrec_requests_max_req_size:

.. confval:: flightrec_requests_max_req_size

Specifies the maximum size (in bytes) of a request entry.
A request entry is truncated if this size is exceeded.
Configuring a flight recorder
-----------------------------

| Type: integer
| Default: 16384
| Environment variable: TT_FLIGHTREC_REQUESTS_MAX_REQ_SIZE
When the flight recorder is enabled, you can set the options related to :ref:`logging <cfg_logging-log>`,
:ref:`metrics <metrics-reference>`, and storing the :ref:`request and response <internals-requests_responses>` data.

The ``flightrec`` configuration might look as follows:

.. _cfg_flightrec_requests_max_res_size:
.. literalinclude:: /code_snippets/snippets/config/instances.enabled/flightrec/config.yaml
:language: yaml
:start-at: flightrec:
:end-at: 10485780
:dedent:

.. confval:: flightrec_requests_max_res_size
In the example, the following options are set:

Specifies the maximum size (in bytes) of a response entry.
A response entry is truncated if this size is exceeded.
* :ref:`flightrec.logs_size <configuration_reference_flightrec_logs_size>` -- a log storage size in bytes.
* :ref:`flightrec.logs_log_level <configuration_reference_flightrec_logs_log_level>` -- a :ref:`log_level <cfg_logging-log_level>`.
* :ref:`flightrec.metrics_period <configuration_reference_flightrec_metrics_period>` -- the number of seconds to store metrics after the dump.
* :ref:`flightrec.metrics_interval <configuration_reference_flightrec_metrics_interval>` -- the frequency of metrics dumps in seconds.
* :ref:`flightrec.requests_size <configuration_reference_flightrec_requests_size>` -- a storage size for the request and response data in bytes.

| Type: integer
| Default: 16384
| Environment variable: TT_FLIGHTREC_REQUESTS_MAX_RES_SIZE
Read more: :ref:`Flight recorder configuration options <configuration_reference_flightrec>`.
8 changes: 4 additions & 4 deletions doc/reference/configuration/cfg_authentication.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies a period of time (in seconds) that a specific user should wait
Specify a period of time (in seconds) that a specific user should wait
for the next attempt after failed authentication.

With the configuration below, Tarantool refuses the authentication attempt if the previous
Expand Down Expand Up @@ -105,7 +105,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the minimum number of characters for a password.
Specify the minimum number of characters for a password.

The following example shows how to set the minimum password length to 10.

Expand Down Expand Up @@ -181,7 +181,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the maximum period of time (in days) a user can use the same password.
Specify the maximum period of time (in days) a user can use the same password.
When this period ends, a user gets the "Password expired" error on a login attempt.
To restore access for such users, use :doc:`box.schema.user.passwd </reference/reference_lua/box_schema/user_passwd>`.

Expand All @@ -207,7 +207,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the number of unique new user passwords before an old password can be reused.
Specify the number of unique new user passwords before an old password can be reused.

In the example below, a new password should differ from the last three passwords.

Expand Down
16 changes: 8 additions & 8 deletions doc/reference/configuration/cfg_flight_recorder.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the size (in bytes) of the log storage.
Specify the size (in bytes) of the log storage.
You can set this option to ``0`` to disable the log storage.

| Type: integer
Expand All @@ -51,7 +51,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the maximum size (in bytes) of the log message.
Specify the maximum size (in bytes) of the log message.
The log message is truncated if its size exceeds this limit.

| Type: integer
Expand All @@ -67,7 +67,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the level of detail the log has.
Specify the level of detail the log has.
You can learn more about log levels from the :ref:`log_level <cfg_logging-log_level>`
option description.
Note that the ``flightrec_logs_log_level`` value might differ from ``log_level``.
Expand All @@ -84,7 +84,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the time period (in seconds) that defines how long metrics are stored from the moment of dump.
Specify the time period (in seconds) that defines how long metrics are stored from the moment of dump.
So, this value defines how much historical metrics data is collected up to the moment of crash.
The frequency of metric dumps is defined by :ref:`flightrec_metrics_interval <cfg_flightrec_metrics_interval>`.

Expand All @@ -100,7 +100,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the time interval (in seconds) that defines the frequency of dumping metrics.
Specify the time interval (in seconds) that defines the frequency of dumping metrics.
This value shouldn't exceed :ref:`flightrec_metrics_period <cfg_flightrec_metrics_period>`.

.. NOTE::
Expand All @@ -125,7 +125,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the size (in bytes) of storage for the request and response data.
Specify the size (in bytes) of storage for the request and response data.
You can set this parameter to ``0`` to disable a storage of requests and responses.

| Type: integer
Expand All @@ -141,7 +141,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the maximum size (in bytes) of a request entry.
Specify the maximum size (in bytes) of a request entry.
A request entry is truncated if this size is exceeded.

| Type: integer
Expand All @@ -156,7 +156,7 @@

Since :doc:`2.11.0 </release/2.11.0>`.

Specifies the maximum size (in bytes) of a response entry.
Specify the maximum size (in bytes) of a response entry.
A response entry is truncated if this size is exceeded.

| Type: integer
Expand Down
Loading