Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
fixup
(cherry picked from commit 7c98e81)
  • Loading branch information
nickanderson committed Jul 23, 2021
1 parent 2602108 commit b45c51c
Showing 1 changed file with 65 additions and 62 deletions.
127 changes: 65 additions & 62 deletions reference/promise-types/measurements.markdown
Expand Up @@ -20,60 +20,6 @@ bundle destined for the agent concerned, in this case `monitor`. However, it is
not necessary to add them to the `bundlesequence`, because `cf-monitord`
executes all bundles of type `monitor`.

Information gathered by `cf-monitord` is stored in `${sys.statedir}/env_data` to
share with other agents like `cf-agent` and `cf-promises` which both provide three
variables per measurement with different prefixes: `value_` for the latest measurement,
`av_` for the running average and `dev_` for the standard deviation of the running average.

The standard measurements and their variable names are listed in [`mon` special variables][mon].

The data over time is stored in `${sys.statedir}/cf_observations.lmdb` which can be examined
most easily with the `cf-check` command:

```
cf-check dump /var/cfengine/state/cf_observations.lmdb
```

By default in the [Masterfiles Policy Framework][Masterfiles Policy Framework], `cf-serverd` uses two variables, `def.default_data_select_host_monitoring_include` and `def.default_data_select_policy_hub_monitoring_include` to [configure which measurements will be included in enterprise reporting][mpf-configure-measurement-collection].

On the hub side, reports are collected and measurements data is inserted into the [`MonitoringHG`][SQL Schema#Table: MonitoringYrMeta] [`MonitoringMgMeta`][SQL Schema#Table: MonitoringMgMeta] and [`MonitoringYrMeta`][SQL Schema#Table: MonitoringYrMeta] tables of the Enterprise Hub database.

A diagnostic query to run with a [Custom Report in Mission Portal][Reporting UI].

Select all unique observables in the database:

```sql
SELECT distinct(observable) FROM monitoringmgmeta;
```

To see which measures are collected for each host, select all data from this
table and then use report result filters to examine a particular host or measure's having
data or not.

```sql
SELECT * FROM monitoringmgmeta;
```

Measurement data is presented in Mission Portal in the [`Measurements App`][Measurements App] and in the ```Measurements``` section of the [`Host Info page`][Hosts#Host Info].

When policy is changed in regards to monitor bundles, both `cf-monitord` _and_ `cf-serverd` should be restarted in order to receive the updated policy.

It is possible to [configure masterfiles to restart `cf-monitord` when variables which affect it's configuration are changed][mpf-configure-component-restart].

All measurements historical data is stored in `${sys.statedir}/cf_observations.lmdb`. This is where reporting data is pulled from.

`${sys.statedir}/env_data` is a current snapshot of measurement values which agents (`cf-agent` and `cf-promises`) use to generate the [special monitor variables][mon].

`${sys.statedir}/ts_key` is the source for the list of observables/measurements used in the system. This file includes the integer id, name and other meta data in a slightly human readable form, for example:

```
0,users,Users with active processes - including system users,average users per 2.5 mins,0.000,100.000,1
```

The `ts_key` file should not be altered.

Note that if a measurement has _always_ had a value of zero it will not be reported and so not available in Mission Portal Measurements or Host Info pages.

```cf3
bundle monitor self_watch
{
Expand All @@ -88,9 +34,9 @@ bundle monitor self_watch
# match_value:
#root \s+ [0-9.]+ \s+ [0-9.]+ \s+ [0-9.]+ \s+ [0-9.]+ \s+ [0-9.]+ \s+ [0-9]+ \s+ [0-9]+ \s+ ([0-9]+) .*"
"/var/cfengine/state/cf_rootprocs"
handle => "cf_monitord_RSS",
stream_type => "file",
data_type => "int",
Expand All @@ -109,12 +55,6 @@ body match_value proc_value(x,y)
}
```

It is important to specify a promise `handle` for measurement promises, as the
names defined in the handle are used to determine the name of the log file or
variable to which data will be reported. Log files are created under
`WORKDIR/state`. Data that have no history type are stored in a special variable
context called `mon`, analogous to the system variables in sys. Thus the values
may be used in other promises in the form `$(mon.handle)`.

```cf3
bundle monitor watch_diskspace
Expand Down Expand Up @@ -147,6 +87,69 @@ what to do with the result afterwards.

* Custom measurements open sourced in 3.12.0

## Architecture

Information gathered by `cf-monitord` is stored in `${sys.statedir}/env_data` to
share with other agents like `cf-agent` and `cf-promises` which both provide three
variables per measurement with different prefixes: `value_` for the latest measurement,
`av_` for the running average and `dev_` for the standard deviation of the running average.

The standard measurements and their variable names are listed in [`mon` special variables][mon].

The data over time is stored in `${sys.statedir}/cf_observations.lmdb` which can be examined
most easily with the `cf-check` command:

```
cf-check dump /var/cfengine/state/cf_observations.lmdb
```

By default in the [Masterfiles Policy Framework][Masterfiles Policy Framework], `cf-serverd` uses two variables, `def.default_data_select_host_monitoring_include` and `def.default_data_select_policy_hub_monitoring_include` to [configure which measurements will be included in enterprise reporting][mpf-configure-measurement-collection].

On the hub side, reports are collected and measurements data is inserted into the [`MonitoringHG`][SQL Schema#Table: MonitoringYrMeta] [`MonitoringMgMeta`][SQL Schema#Table: MonitoringMgMeta] and [`MonitoringYrMeta`][SQL Schema#Table: MonitoringYrMeta] tables of the Enterprise Hub database.

A diagnostic query to run with a [Custom Report in Mission Portal][Reporting UI].

Select all unique observables in the database:

```sql
SELECT distinct(observable) FROM monitoringmgmeta;
```

To see which measures are collected for each host, select all data from this
table and then use report result filters to examine a particular host or measure's having
data or not.

```sql
SELECT * FROM monitoringmgmeta;
```

Measurement data is presented in Mission Portal in the [`Measurements App`][Measurements App] and in the ```Measurements``` section of the [`Host Info page`][Hosts#Host Info].

When policy is changed in regards to monitor bundles, both `cf-monitord` _and_ `cf-serverd` should be restarted in order to receive the updated policy.

It is possible to [configure masterfiles to restart `cf-monitord` when variables which affect it's configuration are changed][mpf-configure-component-restart].

All measurements historical data is stored in `${sys.statedir}/cf_observations.lmdb`. This is where reporting data is pulled from.

`${sys.statedir}/env_data` is a current snapshot of measurement values which agents (`cf-agent` and `cf-promises`) use to generate the [special monitor variables][mon].

`${sys.statedir}/ts_key` is the source for the list of observables/measurements used in the system. This file includes the integer id, name and other meta data in a slightly human readable form, for example:

```
0,users,Users with active processes - including system users,average users per 2.5 mins,0.000,100.000,1
```

The `ts_key` file should not be altered.

Note that if a measurement has _always_ had a value of zero it will not be reported and so not available in Mission Portal Measurements or Host Info pages.

It is important to specify a promise `handle` for measurement promises, as the
names defined in the handle are used to determine the name of the log file or
variable to which data will be reported. Log files are created under
`WORKDIR/state`. Data that have no history type are stored in a special variable
context called `mon`, analogous to the system variables in sys. Thus the values
may be used in other promises in the form `$(mon.handle)`.

***

## Attributes ##
Expand Down

0 comments on commit b45c51c

Please sign in to comment.