Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improving processor error handling/shutdown #353

Merged
merged 3 commits into from
Oct 3, 2021
Merged

Conversation

frairon
Copy link
Contributor

@frairon frairon commented Sep 29, 2021

This PR fixes two main things:

  • Refactoring of error handling to shutdown gracefully or reconnect
  • replace multierr.Errors with hashicorp/go-multierror

replace multierr.Errors with hashicorp/go-multierror
@frairon frairon changed the title Refactoring of error handling to shutdown gracefully or reconnect replace multierr.Errors with hashicorp/go-multierror improving processor error handling/shutdown Sep 30, 2021
errors.go Outdated Show resolved Hide resolved
errors.go Show resolved Hide resolved
partition_processor.go Outdated Show resolved Hide resolved
partition_processor.go Outdated Show resolved Hide resolved
partition_processor.go Outdated Show resolved Hide resolved
partition_processor.go Outdated Show resolved Hide resolved
partition_processor.go Show resolved Hide resolved
partition_processor.go Show resolved Hide resolved
processor.go Outdated Show resolved Hide resolved
errors.go Outdated Show resolved Hide resolved
mockautoconsumers.go Outdated Show resolved Hide resolved
partition_processor.go Outdated Show resolved Hide resolved
partition_processor.go Show resolved Hide resolved
Copy link

@ubunatic ubunatic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (the error handling could cleanup in another PR, if possible at all)

@@ -314,7 +284,7 @@ func (g *Processor) Run(ctx context.Context) (rerr error) {
g.log.Debugf("closing producer")
defer g.log.Debugf("producer ... closed")
if err := g.producer.Close(); err != nil {
merrors.Collect(fmt.Errorf("error closing producer: %v", err))
errs = multierror.Append(errs, fmt.Errorf("error closing producer: %w", err))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting the errs here in this deferred func and then later using the errs to fill the rerr in the previous (but later called) deferred func still twists my head. This whole reversed defer processing makes this really hard to understand.
I think the code is correct, but if there is a cleaner way to write this, the code would be more maintainable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't like to modify the the return value in a deferred function, but haven't found a better solution yet. Maybe we'll find another way.

@frairon frairon merged commit 9aebfe0 into master Oct 3, 2021
@frairon frairon deleted the shutdown-error branch October 3, 2021 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants