New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix notifications log #2218

Merged
merged 1 commit into from Mar 19, 2018

Conversation

Projects
None yet
2 participants
@pradeepbbl
Contributor

pradeepbbl commented Feb 12, 2018

This is follow up to our last PR #2196, we still having difficulties to trace notifications failures due to missing alert key's for action and unknown notifications e.g. mentioned below is an error sample we are getting more often

2018/02/08 16:31:44 error: notify.go:55: sending http: bad response on notification with name memcached_pagerduty for alert method POST: 400

As you can see it's very difficult to find out details like for which alert it's failing and either it's an action notification or an unknown etc, with this change we will be able to get more details e.g

2018/02/09 15:28:06 error: notify.go:56: sending http: bad response for 'memcached_pagerduty' unknown notification using template key '' for alert keys memcached_memory_usage{host=XXXXXXX} method POST: 400

With above we can see the unknown notification is failing due to missing template key and bosun trying to render the default HTML template which in this case will fail because PagerDuty event requires JSON.

Thanks,

Pradeep Mishra
added extra info to POST notifications error log which eases the noti…
…fication failure debugging

	- cmd/bosun/conf/[actionNotify.go, unknownNotify.go]: modified NotificationDetails struct to include temaplte key map and action string
        - cmd/bosun/conf/notify.go: modified PrepareAlert func and notifiaction failure message

@captncraig captncraig self-assigned this Mar 19, 2018

@captncraig captncraig merged commit c733ec4 into bosun-monitor:master Mar 19, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@pradeepbbl pradeepbbl deleted the bookingcom:fix_notifications_log branch Mar 19, 2018

pradeepbbl added a commit to pradeepbbl/bosun that referenced this pull request Mar 21, 2018

added extra info to POST notifications error log which eases the noti…
…fication failure debugging (bosun-monitor#2218)

- cmd/bosun/conf/[actionNotify.go, unknownNotify.go]: modified NotificationDetails struct to include temaplte key map and action string
        - cmd/bosun/conf/notify.go: modified PrepareAlert func and notifiaction failure message

michep pushed a commit to michep/bosun that referenced this pull request Apr 23, 2018

added extra info to POST notifications error log which eases the noti…
…fication failure debugging (bosun-monitor#2218)

- cmd/bosun/conf/[actionNotify.go, unknownNotify.go]: modified NotificationDetails struct to include temaplte key map and action string
        - cmd/bosun/conf/notify.go: modified PrepareAlert func and notifiaction failure message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment