-
Notifications
You must be signed in to change notification settings - Fork 170
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
61 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,24 +1,80 @@ | ||
Prometheus Alertmanager Plugin | ||
============================== | ||
|
||
Silence alerts in Alertmanager when acking alerts in the Alerta console | ||
and delete silences if alerts are manually opened. | ||
Two-way integration with Prometheus which will silence alerts in | ||
Alertmanager when alerts are ack'ed in the Alerta console and delete | ||
silences if those alerts are manually re-opened. | ||
|
||
For help, join [![Gitter chat](https://badges.gitter.im/alerta/chat.png)](https://gitter.im/alerta/chat) | ||
|
||
Installation | ||
------------ | ||
|
||
Clone the GitHub repo and run: | ||
|
||
$ python setup.py install | ||
|
||
Or, to install remotely from GitHub run: | ||
|
||
$ pip install git+https://github.com/alerta/alerta-contrib.git#subdirectory=plugins/prometheus | ||
|
||
Note: If Alerta is installed in a python virtual environment then plugins | ||
need to be installed into the same environment for Alerta to dynamically | ||
discover them. | ||
|
||
Configuration | ||
------------- | ||
|
||
Add `prometheus` to the list of enabled `PLUGINS` in `alertad.conf` | ||
server configuration file and set plugin-specific variables either in | ||
the server configuration file or as environment variables. | ||
|
||
```python | ||
PLUGINS = ['prometheus'] | ||
``` | ||
|
||
The below settings are configured with reasonable defaults: | ||
|
||
```python | ||
ALERTMANAGER_API_URL = 'http://localhost:9093' | ||
ALERTMANAGER_SILENCE_DAYS = 1 | ||
``` | ||
|
||
**Robust Perception Demo Example** | ||
|
||
```python | ||
PLUGINS = ['reject','prometheus'] | ||
ALERTMANAGER_API_URL = 'http://demo.robustperception.io:9093' # default=http://localhost:9093 | ||
ALERTMANAGER_SILENCE_DAYS = 2 # default=1 | ||
``` | ||
|
||
Troubleshooting | ||
--------------- | ||
|
||
Restart Alerta API and confirm that the plugin has been loaded and enabled. | ||
|
||
Set `DEBUG=True` in the `alertad.conf` configuration file and look for log | ||
entries similar to below: | ||
|
||
``` | ||
2016-11-20 23:02:40,623 - alerta.plugins[7394]: DEBUG - Server plug-in 'prometheus' found. [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/alerta_server-4.8.11-py2.7.egg/alerta/plugins/__init__.py:50] | ||
2016-11-20 23:02:40,623 - alerta.plugins[7394]: DEBUG - Server plug-in 'prometheus' not enabled in 'PLUGINS'. [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/alerta_server-4.8.11-py2.7.egg/alerta/plugins/__init__.py:59] | ||
``` | ||
``` | ||
ALERTMANAGER_API_URL = 'https://prom.example.com:9093' # default=http://localhost:9093 | ||
ALERTMANAGER_SILENCE_DAYS = n # default=1 | ||
2016-11-21 10:42:55,572 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: Add silence for alertname=DiskFull instance=web01 [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:35] | ||
2016-11-21 10:42:55,582 - requests.packages.urllib3.connectionpool[8268]: INFO - Starting new HTTP connection (1): demo.robustperception.io [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/requests-2.11.1-py2.7.egg/requests/packages/urllib3/connectionpool.py:214] | ||
2016-11-21 10:42:55,677 - requests.packages.urllib3.connectionpool[8268]: DEBUG - "POST /api/v1/silences HTTP/1.1" 200 44 [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/requests-2.11.1-py2.7.egg/requests/packages/urllib3/connectionpool.py:401] | ||
2016-11-21 10:42:55,711 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: 200 - {"status":"success","data":{"silenceId":29}} [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:59] | ||
2016-11-21 10:42:55,715 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: Added silenceId 29 to attributes [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:67] | ||
``` | ||
|
||
References | ||
---------- | ||
|
||
https://prometheus.io/docs/alerting/alertmanager/#silences | ||
* Alertmanager Silences: https://prometheus.io/docs/alerting/alertmanager/#silences | ||
* Robust Perception On-line Demo: http://demo.robustperception.io:9090/consoles/index.html | ||
|
||
License | ||
------- | ||
|
||
Copyright (c) 2016 Nick Satterly. Available under the MIT License. |