-
Notifications
You must be signed in to change notification settings - Fork 298
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: enabling gateway to ingest events even when sharedDB is down #4262
Conversation
Important Auto Review SkippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the To trigger a single review, invoke the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
bb88d01
to
54b8444
Compare
app/apphandlers/gatewayAppHandler.go
Outdated
@@ -132,23 +132,26 @@ func (a *gatewayApp) StartRudderCore(ctx context.Context, options *app.Options) | |||
}) | |||
drainConfigManager, err := drain_config.NewDrainConfigManager(config, a.log.Child("drain-config")) | |||
if err != nil { | |||
return fmt.Errorf("drain config manager setup: %v", err) | |||
a.log.Errorw("drain config manager setup fail", "error", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So in case of an error, gateway will be operating in degraded mode, at least with respect to draining jobs. Do we plan to setup a P0 alert for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes we will setup an alert for this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice to have the relevant pr mentioned here :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved with non blocking comment
497aee8
to
e830c70
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #4262 +/- ##
==========================================
+ Coverage 73.88% 73.90% +0.02%
==========================================
Files 388 388
Lines 55127 55127
==========================================
+ Hits 40730 40742 +12
+ Misses 12069 12059 -10
+ Partials 2328 2326 -2 ☔ View full report in Codecov by Sentry. |
Description
Gateway was not able to start web server, If sharedDB is down for some reason. Following steps in startup are blocking web server to start.
Solution:
When a new gateway instance is coming up,
We will raise an alert for the failure logs. AI for the alert will be to restart gateway instance.
Linear Ticket
https://linear.app/rudderstack/issue/PIPE-484/gateway-should-accept-data-even-if-it-is-not-able-to-connect-with
Security