An aggregate makes it possible to treat the result of multiple disparate check executions executed across multiple disparate systems as a single result (Event). Aggregates are extremely useful in dynamic environments and/or environments that have a reasonable tolerance for failure. Aggregates should be used when a service can be considered healthy as long as a minimum threshold is satisfied (e.g. are at least 5 healthy web servers? are at least 70% of N processes healthy?).
This plugin allows you to query the Sensu Go Backend API for Events matching certain criteria (labels). This plugin generates a set of counters (e.g. events total, events in an OK state, etc) from the Events query and provides several CLI arguments to evaluate the computed aggregate counters (e.g. --warn-percent=75).
N/A
The Sensu Go Event Aggregates Check plugin
Usage:
sensu-aggregate-check [flags]
Flags:
-H, --api-host string Sensu Go Backend API Host (e.g. 'sensu-backend.example.com') (default "127.0.0.1")
-k, --api-key string Sensu Go Backend API Key
-P, --api-pass string Sensu Go Backend API Password (default "P@ssw0rd!")
-p, --api-port string Sensu Go Backend API Port (e.g. 4242) (default "8080")
-u, --api-user string Sensu Go Backend API User (default "admin")
-l, --check-labels string Sensu Go Event Check Labels to filter by (e.g. 'aggregate=foo')
-C, --crit-count int Critical threshold - count of Events in warning state
-c, --crit-percent int Critical threshold - % of Events in warning state
-e, --entity-labels string Sensu Go Event Entity Labels to filter by (e.g. 'aggregate=foo,app=bar')
-h, --help help for sensu-aggregate-check
-i, --insecure-skip-verify skip TLS certificate verification (not recommended!)
-n, --namespaces string Comma-delimited list of Sensu Go Namespaces to query for Events (e.g. 'us-east-1,us-west-2') (default "default")
-s, --secure Use TLS connection to API
-t, --trusted-ca-file string TLS CA certificate bundle in PEM format
-W, --warn-count int Warning threshold - count of Events in warning state
-w, --warn-percent int Warning threshold - % of Events in warning state
Assets are the best way to make use of this plugin. If you're not using an asset, please consider doing so! If you're using sensuctl 5.13 or later, you can use the following command to add the asset:
sensuctl asset add sensu/sensu-aggregate-check
If you're using an earlier version of sensuctl, you can download the asset definition from this project's Bonsai asset index page or one of the existing releases or create an executable script from this source.
api_version: core/v2
type: CheckConfig
metadata:
namespace: default
name: dummy-app-aggregate
spec:
runtime_assets:
- sensu-aggregate-check
command: sensu-aggregate-check --api-user=foo --api-pass=bar --check-labels='aggregate=healthz,app=dummy' --warn-percent=75 --crit-percent=50
subscriptions:
- backend
publish: true
interval: 30
handlers:
- slack
- pagerduty
- emailIt is advised to use RBAC to create a user scoped specifically for purposes such as this check and to not re-use the admin account. For this check, in particular, the account would need access to list and retrieve events. The example below shows how to create a limited-scope user and the necessary role and role-binding resources to give it the required access.
$ sensuctl user create aggregate --password='4yva#ko!Yq'
Created
$ sensuctl role create get-events --verb list,get --resource events
Created
$ sensuctl role-binding create aggregate-get-events --role=get-events --user=aggregate
Created
Though you could use the user and password combination above with this check, the best practice would be to use an API key instead. You can create the API key with sensuctl:
$ sensuctl api-key grant aggregate
Created: /api/core/v2/apikeys/03f66dbf-6fe0-40d4-8174-95b5eab95649
The key (the text after [...]/apikeys/) above can be used with the --api-key argument in place of using api-user and api-pass.
N/A
To build from source, from the local path of the sensu-aggregate-check repository:
go build
To contribute to this plugin, see CONTRIBUTING