Skip to content
This repository has been archived by the owner. It is now read-only.

Fix PartitionConsumer race condition on Close. #258

Merged
merged 2 commits into from Aug 15, 2018

Conversation

georgeteo
Copy link
Contributor

@georgeteo georgeteo commented Aug 15, 2018

Sarama-cluster calls Sarama Close on Consumer.Close and PartitionConsumer.AsyncClose.

When using the sarama-consumer PartitionConsumer abstraction, this may result in dataloss, because on rebalance or consumer shutdown, consumer application and sarama will both read from the same Messages channel which will result in the application receiving message stream with holes (e.g., offset 100, 102, 104, etc). Since PartitionConsumer application is not synchronously notified of rebalance, this diff changes sarama cluster to use AsyncClose to prevent this dual drain race condition.

dim
dim approved these changes Aug 15, 2018
@dim dim merged commit 6230ffb into bsm:master Aug 15, 2018
@dim
Copy link
Member

dim commented Aug 15, 2018

Thanks, much appreciated

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants