Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log noisy AnimationBucketWarning to firehose, not honeybadger #24778

Merged
merged 3 commits into from Sep 13, 2018

Conversation

davidsbailey
Copy link
Member

The following HB error sometimes happens > 15K times / day: https://app.honeybadger.io/projects/3240/faults/37290310#notice-summary

We want to add a pagerduty alert for anything happening more than 1K/day. So that we don't get paged every day with AnimationBucketWarnings, I'm proposing we move it to Firehose instead.

To verify before merging:

  • verify locally the the firehose event is getting logged.

@davidsbailey
Copy link
Member Author

cc @joshlory

Copy link

@epeach epeach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks for migrating this logging.

@davidsbailey
Copy link
Member Author

I can't quite get a local repro that hits this path, so i've triple-checked the syntax to make sure I think it will meet the requirements needed to make it through to redshift. The repro steps i'm trying are to delete the old versions of an animation via S3 in AWS console, and then using Version History to revert a gamelab project to a version which was using older versions of that animation.

@islemaster
Copy link
Contributor

While you're at it, we should probably move these two as well:

Honeybadger.notify(
error_class: "#{self.class.name}Warning",
error_message: "Restore at Specified Version Failed. Restored most recent.",

Honeybadger.notify(
error_class: "#{self.class.name}Warning",
error_message: "Restore at Specified Version Failed on deleted object. No action taken.",

That could always be a followup PR though.

@davidsbailey davidsbailey merged commit d92766c into staging Sep 13, 2018
@davidsbailey davidsbailey deleted the animation-bucket-warning branch September 13, 2018 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants