yaml-monitoring.rst

Monitoring

The monitor directory can contain two files: monitor/alarmsets.yaml and monitor/logging.yaml. These files contain CloudWatch Alarm and CloudWatch Agent Log Source configuration. These alarms and log sources are grouped into named sets, and sets of alarms and logs can be applied to resources.

Currently only CloudWatch is supported, but it is intended in the future to support other monitoring and logging services in the future.

AlarmSets

Alarm Sets are defined in the file monitor/alarmsets.yaml.

AlarmSets are named to match a Paco Resource type, then a unique AlarmSet name.

The base Alarm schema contains fields to add additional metadata to alarms. For CloudWatchAlarms, this metadata set in the AlarmDescription field as JSON:

Alarms can have different contexts, which increases the number of metadata that is populated in the AlarmDescription field:

• Global context. Only has base context. e.g. a CloudTrail log alarm.
• NetworkEnvironmnet context. Base and NetworkEnvironment context. e.g. a VPC flow log alarm.
• Application context alarm. Base, NetworkEnvironment and Application contexts. e,g, an external HTTP health check alarm
• Resource context alarm. Base, NetworkEnvironment, Application and Resource contexts. e.g. an AutoScalingGroup CPU alarm

Base context for all alarms
----------------------------

"project_name": Project name
"project_title": Project title
"account_name": Account name
"alarm_name": Alarm name
"classification": Classification
"severity": Severity
"topic_arns": SNS Topic ARN subscriptions
"description": Description (only if supplied)
"runbook_url": Runbook URL (only if supplied)

NetworkEnvironment context alarms
---------------------------------

"netenv_name": NetworkEnvironment name
"netenv_title": NetworkEnvironment title
"env_name": Environment name
"env_title": Environment title
"envreg_name": EnvironmentRegion name
"envreg_title": EnvironmentRegion title

Application context alarms
--------------------------

"app_name": Application name
"app_title": Application title

 Resource context alarms
 -----------------------

"resource_group_name": Resource Group name
"resource_group_title": Resource Group title
"resource_name": Resource name
"resource_title": Resource title

Alarms can be set in the monitoring: field for Application and Resource objects. The name of each AlarmSet should be listed in the alarm_sets: field. It is possible to override the individual fields of an Alarm in a netenv file.

Stylistically, monitoring and alarm_sets can be specified in the base applications: section in a netenv file, and set to enabled: false. Then only the production environment can override the enabled field to true. This makes it easy to enable a dev or test environment if you want to test alarms before using in a production environment.

Alternatively, you may wish to only specify the monitoring in the environments: section of your netenv file only for production, and keep the base applications: configuration shorter.

Alarm notifications tell alarms which SNS Topics to notify. Alarm notifications are set with the notifications: field at the Application, Resource, AlarmSet and Alarm level.

Notifications can be filtered for specific severity and classification levels. This allows you to direct critical severity to one group and low severity to another, or to send only performance classification alarms to one group and security classification alarms to another.

Note that although you can configure multiple SNS Topics to subscribe to a single alarm, CloudWatch has a maximum limit of five SNS Topics that a given alarm may be subscribed to.

It is also possible to write a Paco add-on that overrides the default CloudWatch notifications and instead notifies a single SNS Topic. This is intended to allow you to write an add-on that directs all alarms through a single Lambda (regardless or account or region) which is then responsible for delivering or taking action on alarms.

Currently Global and NetworkEnvironment alarms are only supported through Paco add-ons.

:guilabel:`AlarmSets`|bars| Container<AlarmSet>

Field name	Type	Purpose	Constraints	Default

Base Schemas Named, Title

AlarmSet

A container of Alarm objects.

:guilabel:`AlarmSet`

Field name	Type	Purpose	Constraints	Default
resource_type	String	Resource type	Must be a valid AWS resource type

Base Schemas Named, Notifiable, Title

Alarm

A Paco Alarm.

This is a base schema which defines metadata useful to categorize an alarm.

:guilabel:`Alarm`

Field name	Type	Purpose	Constraints	Default
classification	String |star|	Classification	Must be one of: 'performance', 'security' or 'health'	unset
description	String	Description
notification_groups	List<String>	List of notification groups the alarm is subscribed to.
runbook_url	String	Runbook URL
severity	String	Severity	Must be one of: 'low', 'critical'	low

Base Schemas Deployable, Named, Notifiable, Title

Dimension

A dimension of a metric

:guilabel:`Dimension`

Field name	Type	Purpose	Constraints	Default
name	String	Dimension name
value	PacoReference|String	String or a Paco Reference to resource output.	Paco Reference to Interface. String Ok.

AlarmNotifications

Container for AlarmNotification objects.

:guilabel:`AlarmNotifications`|bars| Container<AlarmNotification>

Field name	Type	Purpose	Constraints	Default

Base Schemas Named, Title

AlarmNotification

Alarm Notification

:guilabel:`AlarmNotification`

Field name	Type	Purpose	Constraints	Default
classification	String	Classification filter	Must be one of: 'performance', 'security', 'health' or ''.
groups	List<String> |star|	List of groups
severity	String	Severity filter	Must be one of: 'low', 'critical'

Base Schemas Named, Title

SimpleCloudWatchAlarm

A Simple CloudWatch Alarm

:guilabel:`SimpleCloudWatchAlarm`

Field name	Type	Purpose	Constraints	Default
actions_enabled	Boolean	Actions Enabled
alarm_description	String	Alarm Description	Valid JSON document with Paco fields.
comparison_operator	String	Comparison operator	Must be one of: 'GreaterThanThreshold','GreaterThanOrEqualToThreshold', 'LessThanThreshold', 'LessThanOrEqualToThreshold'
dimensions	List<Dimension>	Dimensions
evaluation_periods	Int	Evaluation periods
metric_name	String |star|	Metric name
namespace	String	Namespace
period	Int	Period in seconds
statistic	String	Statistic
threshold	Float	Threshold

MetricFilters

Container for Metric`Filter objects.

:guilabel:`MetricFilters`|bars| Container<MetricFilter>

Field name	Type	Purpose	Constraints	Default

Base Schemas Named, Title

MetricFilter

Metric filter

:guilabel:`MetricFilter`

Field name	Type	Purpose	Constraints	Default
filter_pattern	String	Filter pattern
metric_transformations	List<MetricTransformation>	Metric transformations

Base Schemas Named, Title

MetricTransformation

Metric Transformation

:guilabel:`MetricTransformation`

Field name	Type	Purpose	Constraints	Default
default_value	Float	The value to emit when a filter pattern does not match a log event.
metric_name	String |star|	The name of the CloudWatch Metric.
metric_namespace	String	The namespace of the CloudWatch metric. If not set, the namespace used will be 'AIM/{log-group-name}'.
metric_value	String |star|	The value that is published to the CloudWatch metric.

Metric

A set of metrics to collect and an optional collection interval:

• name: disk

  measurements: - free collection_interval: 900

:guilabel:`Metric`

Field name	Type	Purpose	Constraints	Default
collection_interval	Int	Collection interval
drop_device	Boolean	Drops the device name from disk metrics		True
measurements	List<String>	Measurements
name	String	Metric(s) group name
resources	List<String>	List of resources for this metric

CloudWatchLogging

CloudWatch Logging configuration

:guilabel:`CloudWatchLogging`

Field name	Type	Purpose	Constraints	Default
log_sets	Container<CloudWatchLogSets>	A CloudWatchLogSets container

Base Schemas CloudWatchLogRetention, Named, Title

CloudWatchLogRetention

:guilabel:`CloudWatchLogRetention`

Field name	Type	Purpose	Constraints	Default
expire_events_after_days	String	Expire Events After. Retention period of logs in this group

CloudWatchLogSets

Container for CloudWatchLogSet objects.

:guilabel:`CloudWatchLogSets`|bars| Container<CloudWatchLogSet>

Field name	Type	Purpose	Constraints	Default

Base Schemas Named, Title

CloudWatchLogSet

A set of Log Group objects

:guilabel:`CloudWatchLogSet`

Field name	Type	Purpose	Constraints	Default
log_groups	Container<CloudWatchLogGroups>	A CloudWatchLogGroups container

Base Schemas CloudWatchLogRetention, Named, Title

CloudWatchLogGroups

Container for CloudWatchLogGroup objects.

:guilabel:`CloudWatchLogGroups`|bars| Container<CloudWatchLogGroup>

Field name	Type	Purpose	Constraints	Default

Base Schemas Named, Title

CloudWatchLogGroup

A CloudWatchLogGroup is responsible for retention, access control and metric filters

:guilabel:`CloudWatchLogGroup`

Field name	Type	Purpose	Constraints	Default
log_group_name	String	Log group name. Can override the LogGroup name used from the name field.
metric_filters	Container<MetricFilters>	Metric Filters
sources	Container<CloudWatchLogSources>	A CloudWatchLogSources container

Base Schemas CloudWatchLogRetention, Named, Title

CloudWatchLogSources

A container of CloudWatchLogSource objects.

:guilabel:`CloudWatchLogSources`|bars| Container<CloudWatchLogSource>

Field name	Type	Purpose	Constraints	Default

Base Schemas Named, Title

CloudWatchLogSource

Log source for a CloudWatch agent.

:guilabel:`CloudWatchLogSource`

Field name	Type	Purpose	Constraints	Default
encoding	String	Encoding		utf-8
log_stream_name	String |star|	Log stream name	CloudWatch Log Stream name
multi_line_start_pattern	String	Multi-line start pattern
path	String |star|	Path	Must be a valid filesystem path expression. Wildcard * is allowed.
timestamp_format	String	Timestamp format
timezone	String	Timezone	Must be one of: 'Local', 'UTC'	Local

Base Schemas CloudWatchLogRetention, Named, Title

HealthChecks

Container for Route53HealthCheck objects.

:guilabel:`HealthChecks`|bars| Container<Route53HealthCheck>

Field name	Type	Purpose	Constraints	Default

Base Schemas Named, Title