Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure alert limiter broken #2074

Closed
taylordowns2000 opened this issue May 11, 2024 · 4 comments
Closed

Failure alert limiter broken #2074

taylordowns2000 opened this issue May 11, 2024 · 4 comments
Assignees
Labels
bug Newly identified bug

Comments

@taylordowns2000
Copy link
Member

taylordowns2000 commented May 11, 2024

FunctionClauseError no function clause matching in Thunderbolt.Extensions.UsageLimiter.limit_action/2

FunctionClauseError: no function clause matching in Thunderbolt.Extensions.UsageLimiter.limit_action/2
  File "lib/thunderbolt/extensions/usage_limiter.ex", line 30, in Thunderbolt.Extensions.UsageLimiter.limit_action/2
  File "lib/lightning/pipeline/failure_alerter.ex", line 17, in Lightning.FailureAlerter.alert_on_failure/1
  File "lib/lightning_web/channels/run_channel.ex", line 95, in LightningWeb.RunChannel.handle_in/3
  File "lib/phoenix/channel/server.ex", line 322, in Phoenix.Channel.Server.handle_info/2
  File "gen_server.erl", line 1095, in :gen_server.try_handle_info/3

https://openfn.sentry.io/issues/5321698515/?project=6446500&query=is%3Aunresolved&referrer=issue-stream&statsPeriod=30d&stream_index=1&utc=true

@taylordowns2000 taylordowns2000 added the bug Newly identified bug label May 11, 2024
Copy link

sentry-io bot commented May 12, 2024

Sentry Issue: LIGHTNING-CA

@midigofrank midigofrank self-assigned this May 13, 2024
@midigofrank
Copy link
Collaborator

I'm unable to reproduce this locally on thunderbolt's main. Which version of thunderbolt do we have running in prod?
cc: @taylordowns2000 @stuartc

@taylordowns2000
Copy link
Member Author

still blowing up regularly on production: app:v0.3.0-wte https://openfn.sentry.io/issues/5321698515/?project=6446500&referrer=github_integration

@midigofrank , @christad92 , @stuartc , does this mean that:

  1. we're actually crashing for all these projects? is this async? what happens downstream of this error? how critical is it that we blow up?
  2. we're failing to send failure alerts when runs fail for all projects?
  3. we're alerting properly for paid projects but getting this exception for free plan projects?

or something else?

@midigofrank
Copy link
Collaborator

Hey @taylordowns2000

  1. Yes we are crashing for all projects. The only bit affected by this blow up is the email being sent out. The run is first marked complete successfully so the run isn't affected.
  2. Yes. No project gets the failure alerts with this error
  3. No. this fails for all projects regardless

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Newly identified bug
Projects
Archived in project
Development

No branches or pull requests

2 participants