Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix notifications log #2218

Merged
merged 1 commit into from
Mar 19, 2018

Conversation

pradeepbbl
Copy link
Contributor

@pradeepbbl pradeepbbl commented Feb 12, 2018

This is follow up to our last PR #2196, we still having difficulties to trace notifications failures due to missing alert key's for action and unknown notifications e.g. mentioned below is an error sample we are getting more often

2018/02/08 16:31:44 error: notify.go:55: sending http: bad response on notification with name memcached_pagerduty for alert method POST: 400

As you can see it's very difficult to find out details like for which alert it's failing and either it's an action notification or an unknown etc, with this change we will be able to get more details e.g

2018/02/09 15:28:06 error: notify.go:56: sending http: bad response for 'memcached_pagerduty' unknown notification using template key '' for alert keys memcached_memory_usage{host=XXXXXXX} method POST: 400

With above we can see the unknown notification is failing due to missing template key and bosun trying to render the default HTML template which in this case will fail because PagerDuty event requires JSON.

Thanks,

@pradeepbbl pradeepbbl force-pushed the fix_notifications_log branch from 0b21b7b to 36c0f71 Compare February 21, 2018 15:18
…fication failure debugging

	- cmd/bosun/conf/[actionNotify.go, unknownNotify.go]: modified NotificationDetails struct to include temaplte key map and action string
        - cmd/bosun/conf/notify.go: modified PrepareAlert func and notifiaction failure message
@pradeepbbl pradeepbbl force-pushed the fix_notifications_log branch from 36c0f71 to 6c53990 Compare February 21, 2018 15:25
@captncraig captncraig self-assigned this Mar 19, 2018
@captncraig captncraig merged commit c733ec4 into bosun-monitor:master Mar 19, 2018
@pradeepbbl pradeepbbl deleted the fix_notifications_log branch March 19, 2018 17:35
pradeepbbl added a commit to pradeepbbl/bosun that referenced this pull request Mar 21, 2018
…fication failure debugging (bosun-monitor#2218)

- cmd/bosun/conf/[actionNotify.go, unknownNotify.go]: modified NotificationDetails struct to include temaplte key map and action string
        - cmd/bosun/conf/notify.go: modified PrepareAlert func and notifiaction failure message
michep pushed a commit to michep/bosun that referenced this pull request Apr 23, 2018
…fication failure debugging (bosun-monitor#2218)

- cmd/bosun/conf/[actionNotify.go, unknownNotify.go]: modified NotificationDetails struct to include temaplte key map and action string
        - cmd/bosun/conf/notify.go: modified PrepareAlert func and notifiaction failure message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants