Skip to content
This repository has been archived by the owner on Feb 3, 2023. It is now read-only.

Commit

Permalink
KPI base searches
Browse files Browse the repository at this point in the history
  • Loading branch information
guilhemmarchand committed Aug 4, 2021
1 parent 7ed09bd commit 17ba8a9
Show file tree
Hide file tree
Showing 9 changed files with 126 additions and 40 deletions.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics1.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics2.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics3.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics4.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics5.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics6.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics7.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/itsi_v2/kpi_basesearch_metrics8.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
166 changes: 126 additions & 40 deletions docs/itsi_integration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -96,11 +96,13 @@ ITSI entities import
:alt: itsi_entities3.png
:align: center
:width: 1200px
:class: with-border

.. image:: img/itsi_entities4.png
:alt: itsi_entities4.png
:align: center
:width: 1200px
:class: with-border

Any new data source discovered and configured in TrackMe will be created in ITSI, and existing entities will be maintained automatically.

Expand All @@ -109,85 +111,169 @@ Step 2: create the KPI base search for metrics

**The next step is to create a KPI base saarch that will turn the metrics into ITSI KPIs, within the KPI base search editor, create a new base search:**

.. image:: img/itsi_kpi1.png
:alt: itsi_kpi1.png
:align: center
:width: 1200px
*KPI base search title and description:*

- Title: **TrackMe:Metrics**
- Description: **This KPI base search handles TrackMe metrics for data sources monitoring**

**We will create a KPI base search handling the metric index, we use an adhoc search instead of the Metric Search type to allow one KPI base search to handle all of the metrics at once:**
*KPI base search*:

*Ad-hoc search:*
- Type: **adhoc**
- search:

::

| mstats latest(trackme.eventcount_4h) as trackme.eventcount_4h, latest(trackme.lag_event_sec) as trackme.lag_event_sec, latest(trackme.lag_ingestion_sec) as trackme.lag_ingestion_sec where index=trackme_metrics by object_category, object
| mstats latest(trackme.eventcount_4h) as trackme.eventcount_4h, latest(trackme.lag_event_sec) as trackme.lag_event_sec, latest(trackme.lag_ingestion_sec) as trackme.lag_ingestion_sec where index=trackme_metrics by object_category, object

- KPI Search Schedule: **Every 5 minutes**
- Calculation Window: **Last 5 minutes**
- Monitoring Lag (in seconds): **30**
- Split by Entity: **yes**
- Entity Split Field: **object**
- Filter to Entities in Service: **yes**
- Entity Filter Field: **object**

.. image:: img/itsi_kpi2.png
:alt: itsi_kpi2.png
.. image:: img/itsi_v2/kpi_basesearch_metrics1.png
:alt: kpi_basesearch_metrics1.png
:align: center
:width: 1200px
:class: with-border

*Then, create the metrics as follows:*

**Metric: trackme.eventcount_4h**

*Then add each metric as a new KPI: (see the last screen for the definition of the calculation and service level aggregation)*
- Title: **eventcount_4h**
- threshold field: **trackme.eventcount_4h**
- Unit: **#**
- Entity calculation: **Average**
- Service / Aggregate calculation: **Sum**
- Fill Data Gaps with: **Null values**
- Threshold level for Null values: **Unknown**

.. image:: img/itsi_kpi3.png
:alt: itsi_kpi3.png
.. image:: img/itsi_v2/kpi_basesearch_metrics2.png
:alt: kpi_basesearch_metrics2.png
:align: center
:width: 1200px
:width: 500px
:class: with-border

.. image:: img/itsi_kpi4.png
:alt: itsi_kpi4.png
**Metric: trackme.lag_event_sec**

- Title: **lag_event_sec**
- threshold field: **trackme.lag_event_sec**
- Unit: **sec**
- Entity calculation: **Average**
- Service / Aggregate calculation: **Average**
- Fill Data Gaps with: **Null values**
- Threshold level for Null values: **Unknown**

.. image:: img/itsi_v2/kpi_basesearch_metrics3.png
:alt: kpi_basesearch_metrics3.png
:align: center
:width: 1200px
:width: 500px
:class: with-border

**Metric: trackme.lag_ingestion_sec**

- Title: **lag_ingestion_sec**
- threshold field: **trackme.lag_ingestion_sec**
- Unit: **sec**
- Entity calculation: **Average**
- Service / Aggregate calculation: **Average**
- Fill Data Gaps with: **Null values**
- Threshold level for Null values: **Unknown**

.. image:: img/itsi_v2/kpi_basesearch_metrics4.png
:alt: kpi_basesearch_metrics4.png
:align: center
:width: 500px
:class: with-border

Step 3: create the KPI base searches for summary statuses events
----------------------------------------------------------------

**TrackMe automatically generates statuses summary events every time trackers are executed.**
*KPI base search title and description:*

**These summary events can be used by ITSI to detect changes in the state of TrackMe entities and feed ITSI episodes:**
- Title: **TrackMe:FlappingStatuses**
- Description: **This KPI base searches handles TrackMe status flapping events**

*Create three new KPI base searches:*
*KPI base search:*

*data sources tracking*
- Type: **adhoc**
- search:

::

`trackme_idx` object_category="data_source" source="current_state_tracking:data_source" priority=*
`trackme_idx` source="current_state_tracking:*" priority=*
| eval {priority}_{current_state} = current_state

*data hosts tracking*

::
- KPI Search Schedule: **Every 5 minutes**
- Calculation Window: **Last 5 minutes**
- Monitoring Lag (in seconds): **60**
- Split by Entity: **yes**
- Entity Split Field: **object**
- Filter to Entities in Service: **yes**
- Entity Filter Field: **object**

`trackme_idx` object_category="data_host" source="current_state_tracking:data_host" priority=*
| eval {priority}_{current_state} = current_state
.. image:: img/itsi_v2/kpi_basesearch_metrics5.png
:alt: kpi_basesearch_metrics5.png
:align: center
:width: 1200px
:class: with-border

*metric hosts tracking*
*Then, create the metrics as follows:*

::
**Metric: high_red**

`trackme_idx` object_category="metric_host" source="current_state_tracking:metric_host" priority=*
| eval {priority}_{current_state} = current_state
- Title: **high_red**
- threshold field: **high_red**
- Unit: **#**
- Entity calculation: **Count**
- Service / Aggregate calculation: **Sum**
- Fill Data Gaps with: **Null values**
- Threshold level for Null values: **Normal**

.. image:: img/itsi_summary_events_kpi_data_source.png
:alt: iitsi_summary_events_kpi_data_source.png
.. image:: img/itsi_v2/kpi_basesearch_metrics6.png
:alt: kpi_basesearch_metrics6.png
:align: center
:width: 1200px
:width: 500px
:class: with-border

**Metric: medium_red**

.. image:: img/itsi_summary_events_kpi_data_host.png
:alt: iitsi_summary_events_kpi_data_host.png
- Title: **medium_red**
- threshold field: **medium_red**
- Unit: **#**
- Entity calculation: **Count**
- Service / Aggregate calculation: **Sum**
- Fill Data Gaps with: **Null values**
- Threshold level for Null values: **Normal**

.. image:: img/itsi_v2/kpi_basesearch_metrics7.png
:alt: kpi_basesearch_metrics7.png
:align: center
:width: 1200px
:width: 500px
:class: with-border

**Metric: low_red**

- Title: **low_red**
- threshold field: **low_red**
- Unit: **#**
- Entity calculation: **Count**
- Service / Aggregate calculation: **Sum**
- Fill Data Gaps with: **Null values**
- Threshold level for Null values: **Normal**

.. image:: img/itsi_summary_events_kpi_metric_host.png
:alt: iitsi_summary_events_kpi_metric_host.png
.. image:: img/itsi_v2/kpi_basesearch_metrics8.png
:alt: kpi_basesearch_metrics8.png
:align: center
:width: 1200px
:width: 500px
:class: with-border

*Notes:*

- the technique ``{priority}_{current_state} = current_state`` allows you to track different levels of priorities easily without any conditional operations
- these examples above assume you monitor in ITSI only high priority entities, repeat these operations if you want to track other levels of priorities

Step 4: create a service that will be used for the service template definition
------------------------------------------------------------------------------
Expand Down

0 comments on commit 17ba8a9

Please sign in to comment.