Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix custom consumer group error channel lifecycle #263

Merged
merged 3 commits into from Dec 8, 2020

Conversation

lionelvillard
Copy link
Contributor

Fixes #174

Proposed Changes

  • Scope consumer group handler to a session.
  • Scope error channel to a consumer group

Release Note

🐛 Fix crash in Kafka consumer when a rebalance occurs

Docs

@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Dec 8, 2020
@knative-prow-robot knative-prow-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Dec 8, 2020
@knative-prow-robot knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 8, 2020
@codecov
Copy link

codecov bot commented Dec 8, 2020

Codecov Report

Merging #263 (1a1f566) into master (042279b) will decrease coverage by 0.01%.
The diff coverage is 87.50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #263      +/-   ##
==========================================
- Coverage   75.68%   75.67%   -0.02%     
==========================================
  Files         114      114              
  Lines        4479     4481       +2     
==========================================
+ Hits         3390     3391       +1     
- Misses        887      888       +1     
  Partials      202      202              
Impacted Files Coverage Δ
pkg/common/consumer/consumer_factory.go 75.75% <80.00%> (-1.67%) ⬇️
pkg/common/consumer/consumer_handler.go 80.95% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 042279b...db90916. Read the comment docs.

@lberk
Copy link
Member

lberk commented Dec 8, 2020

awesome! thanks for putting this together. Given the nature of the crash -- is there any way we could add an e2e test to make sure we catch this moving forward? Delete the sink during a session?

@lionelvillard
Copy link
Contributor Author

I still need to find a good way to test this. The problem with deleting the sink is it takes 4-5mn for the buffer to be depleted (not sure exactly why).

The way to reproduce is:

  1. send events to a broken sink
  2. trigger a rebalance by changing the number of replicas on the adapter
  3. wait a bit for ConsumerHandler to Cleanup
  4. a new consumer handler is created (no crash)

@knative-prow-robot knative-prow-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Dec 8, 2020
@knative-metrics-robot
Copy link

The following is the coverage report on the affected files.
Say /test pull-knative-sandbox-eventing-kafka-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/common/consumer/consumer_factory.go 83.9% 81.8% -2.1

@lberk
Copy link
Member

lberk commented Dec 8, 2020

/lgtm
/approve

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Dec 8, 2020
@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lberk, lionelvillard

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [lberk,lionelvillard]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pierDipi
Copy link
Member

pierDipi commented Dec 8, 2020

/cherrypick release-0.19

@knative-prow-robot
Copy link
Contributor

@pierDipi: new pull request created: #264

In response to this:

/cherrypick release-0.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

lionelvillard added a commit to lionelvillard/eventing-kafka that referenced this pull request Dec 8, 2020
…#263)

* fix custom consumer group error channel lifecycle

* fix import

* remove unused var
matzew pushed a commit to matzew/eventing-kafka that referenced this pull request Aug 24, 2021
…e-extensions#263)

catch branches on openshift that match v0.X and the previous v.X.Y scheme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Indicates the PR's author has signed the CLA. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

KafkaSource dispatcher crashes when sink is deleted (and session closed?)
5 participants