too many events notice when I don't intend to use events

[wlipa]

With the latest honeybadger gem in a Rails app, I'm seeing lots of log messages like:

  ** [Honeybadger] Event send failed: project is sending too many events. code=429 throttle=41 interval=6.392 level=2 pid=10094
  ** [Honeybadger] Event send failed: project is sending too many events. code=429 throttle=124 interval=423.105 level=2 pid=10077

I am not familiar with events and never turned them on as far as I know. So I'm surprised to see this message.

[stympy]

Sorry for the inconvenience! We enabled events by default in v6 of the gem. You can disable it by adding this to honeybadger.yml:

insights:
  enabled: false

More information about events configuration can be found here.

You're also welcome to test the reduce-events-logging branch if you want to try Insights and not get so much logging once you hit the daily data limit.

[wlipa]

OK, wow. That is a little surprising to have turned on by default. I reviewed the change log for the gem, and it's not that apparent that a gem upgrade will cause a flood of data to start exfiltrating from our apps. Simple fix though, but I would have expected a more overt notification.

[Nathan-Heller]

yeah, I am a little frustrated this was enabled by default, its weird to have this enabled by default, and then get pinged for rate limit problems.

this ended up causing issues in our deployment pipeline, because honeybadger's 429 would prevent our Fargate ECS instances from successfully booting up.

[stympy]

Version 6.0.6 of the gem has been released — it changes the log level from warn to debug for this log message.

[wlipa]

The surprising thing is less the logging and more that the feature is turned on with little warning.

[Nathan-Heller]

agreed

we have x-ray/open telemetry, and this feature shows up as an entirely different node in the trace map, because (i think) it spins up as a different worker.

i couldnt for the life of me figure out why, until this showed up in our failed ECS instance boot logs.

[stympy]

What would have been helpful to you beyond the major version change and the changelog?

[wlipa]

Instead of:

enable Insights by default

how about:

enable Insights by default. Upgrading to this version will cause your app to start sending large numbers of non-error status/informational events to Honeybadger, so configure this off if you don't want that.

Basically the issue is I don't follow things closely enough to know what Insights is just by looking at that.