Permalink
Browse files

Merge pull request #8 from obfuscurity/pagerduty

Support for PagerDuty incidents
  • Loading branch information...
2 parents 29b985c + 2f849a9 commit 153e0594049171157f75f5c310de41ab21b390c1 @obfuscurity committed Apr 6, 2013
Showing with 43 additions and 2 deletions.
  1. +2 −1 Gemfile
  2. +2 −1 Gemfile.lock
  3. +10 −0 README.md
  4. +29 −0 lib/backstop/web.rb
View
@@ -1,5 +1,6 @@
-source :rubygems
+source 'https://rubygems.org'
+gem 'bundler'
gem 'sinatra'
gem 'thin'
gem 'json'
View
@@ -1,5 +1,5 @@
GEM
- remote: http://rubygems.org/
+ remote: https://rubygems.org/
specs:
daemons (1.1.8)
diff-lcs (1.1.3)
@@ -35,6 +35,7 @@ PLATFORMS
ruby
DEPENDENCIES
+ bundler
json
newrelic_rpm
rack-test
View
@@ -30,6 +30,16 @@ All GitHub commit metrics contain the project name, branch information, author e
github.project.refs.heads.master.bob-example-com.10af2cb02eadd4cb1a3e43aa9cae47ef2cd07016 1 1203116237
```
+### PagerDuty Incident Webhooks
+
+Backstop can also receive PagerDuty incidents courtesy of Jesse Newland's [pagerduty-incident-webhooks](https://github.com/github/pagerduty-incident-webhooks) project. When deploying `pagerduty-incident-webhooks` make sure to set `PAGERDUTY_WEBHOOK_ENDPOINT` to your Backstop service URL with the `/pagerduty` endpoint. For example, `https://backstop.example.com/pagerduty`.
+
+Metrics will be stored under the `alerts` prefix with a value of `1`. These can then be visualized as annotation-style metrics using Graphite's `drawAsFinite()` function. Sample metric:
+
+```
+alerts.nagios.web1.diskspace 1 1365206103
+```
+
### Custom Metrics
Use the `/publish` endpoint in conjunction with one of the approved `PREFIXES` for submitting metrics to Backstop. In most environments it makes sense to use distinct prefixes for normal (e.g. gauge, counters, etc) metrics vs annotation (event-style) metrics.
View
@@ -1,5 +1,6 @@
require 'sinatra'
require 'json'
+require 'time'
require 'backstop'
@@ -57,6 +58,34 @@ def publisher
'ok'
end
+ post '/pagerduty' do
+ begin
+ incident = params
+ rescue
+ halt 400, 'unknown payload'
+ end
+ case incident['service']['name']
+ when 'Pingdom'
+ metric = "pingdom.#{incident['incident_key'].gsub(/\./, '_').gsub(/[\(\)]/, '').gsub(/\s+/, '.')}"
+ when 'nagios'
+ data = incident['trigger_summary_data']
+ outage = data['SERVICEDESC'] === '' ? 'host_down' : data['SERVICEDESC']
+ begin
+ metric = "nagios.#{data['HOSTNAME'].gsub(/\./, '_')}.#{outage}"
+ rescue
+ puts "UNKNOWN ALERT: #{incident.to_json}"
+ halt 400, 'unknown alert'
+ end
+ when 'Enterprise Zendesk'
+ metric = "enterprise.zendesk.#{incident['service']['id']}"
+ else
+ puts "UNKNOWN ALERT: #{incident.to_json}"
+ halt 400, 'unknown alert'
+ end
+ publisher.publish("alerts.#{metric}", 1, Time.parse(incident['created_on']).to_i)
+ 'ok'
+ end
+
post '/publish/:name' do
begin
data = JSON.parse(request.body.read)

0 comments on commit 153e059

Please sign in to comment.