Skip to content

Commit

Permalink
Add more info to Prometheus README
Browse files Browse the repository at this point in the history
  • Loading branch information
satterly committed Nov 21, 2016
1 parent 9799a79 commit 085846e
Showing 1 changed file with 61 additions and 5 deletions.
66 changes: 61 additions & 5 deletions plugins/prometheus/README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,80 @@
Prometheus Alertmanager Plugin
==============================

Silence alerts in Alertmanager when acking alerts in the Alerta console
and delete silences if alerts are manually opened.
Two-way integration with Prometheus which will silence alerts in
Alertmanager when alerts are ack'ed in the Alerta console and delete
silences if those alerts are manually re-opened.

For help, join [![Gitter chat](https://badges.gitter.im/alerta/chat.png)](https://gitter.im/alerta/chat)

Installation
------------

Clone the GitHub repo and run:

$ python setup.py install

Or, to install remotely from GitHub run:

$ pip install git+https://github.com/alerta/alerta-contrib.git#subdirectory=plugins/prometheus

Note: If Alerta is installed in a python virtual environment then plugins
need to be installed into the same environment for Alerta to dynamically
discover them.

Configuration
-------------

Add `prometheus` to the list of enabled `PLUGINS` in `alertad.conf`
server configuration file and set plugin-specific variables either in
the server configuration file or as environment variables.

```python
PLUGINS = ['prometheus']
```

The below settings are configured with reasonable defaults:

```python
ALERTMANAGER_API_URL = 'http://localhost:9093'
ALERTMANAGER_SILENCE_DAYS = 1
```

**Robust Perception Demo Example**

```python
PLUGINS = ['reject','prometheus']
ALERTMANAGER_API_URL = 'http://demo.robustperception.io:9093' # default=http://localhost:9093
ALERTMANAGER_SILENCE_DAYS = 2 # default=1
```

Troubleshooting
---------------

Restart Alerta API and confirm that the plugin has been loaded and enabled.

Set `DEBUG=True` in the `alertad.conf` configuration file and look for log
entries similar to below:

```
2016-11-20 23:02:40,623 - alerta.plugins[7394]: DEBUG - Server plug-in 'prometheus' found. [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/alerta_server-4.8.11-py2.7.egg/alerta/plugins/__init__.py:50]
2016-11-20 23:02:40,623 - alerta.plugins[7394]: DEBUG - Server plug-in 'prometheus' not enabled in 'PLUGINS'. [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/alerta_server-4.8.11-py2.7.egg/alerta/plugins/__init__.py:59]
```
```
ALERTMANAGER_API_URL = 'https://prom.example.com:9093' # default=http://localhost:9093
ALERTMANAGER_SILENCE_DAYS = n # default=1
2016-11-21 10:42:55,572 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: Add silence for alertname=DiskFull instance=web01 [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:35]
2016-11-21 10:42:55,582 - requests.packages.urllib3.connectionpool[8268]: INFO - Starting new HTTP connection (1): demo.robustperception.io [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/requests-2.11.1-py2.7.egg/requests/packages/urllib3/connectionpool.py:214]
2016-11-21 10:42:55,677 - requests.packages.urllib3.connectionpool[8268]: DEBUG - "POST /api/v1/silences HTTP/1.1" 200 44 [in /var/lib/.virtualenvs/alerta/lib/python2.7/site-packages/requests-2.11.1-py2.7.egg/requests/packages/urllib3/connectionpool.py:401]
2016-11-21 10:42:55,711 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: 200 - {"status":"success","data":{"silenceId":29}} [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:59]
2016-11-21 10:42:55,715 - alerta.plugins.prometheus[8268]: DEBUG - Alertmanager: Added silenceId 29 to attributes [in build/bdist.macosx-10.12-x86_64/egg/alerta_prometheus.py:67]
```

References
----------

https://prometheus.io/docs/alerting/alertmanager/#silences
* Alertmanager Silences: https://prometheus.io/docs/alerting/alertmanager/#silences
* Robust Perception On-line Demo: http://demo.robustperception.io:9090/consoles/index.html

License
-------

Copyright (c) 2016 Nick Satterly. Available under the MIT License.

0 comments on commit 085846e

Please sign in to comment.