Skip to content
This repository has been archived by the owner. It is now read-only.

Fix PartitionConsumer race condition on Close. #258

merged 2 commits into from Aug 15, 2018


Copy link

@georgeteo georgeteo commented Aug 15, 2018

Sarama-cluster calls Sarama Close on Consumer.Close and PartitionConsumer.AsyncClose.

When using the sarama-consumer PartitionConsumer abstraction, this may result in dataloss, because on rebalance or consumer shutdown, consumer application and sarama will both read from the same Messages channel which will result in the application receiving message stream with holes (e.g., offset 100, 102, 104, etc). Since PartitionConsumer application is not synchronously notified of rebalance, this diff changes sarama cluster to use AsyncClose to prevent this dual drain race condition.

dim approved these changes Aug 15, 2018
@dim dim merged commit 6230ffb into bsm:master Aug 15, 2018
Copy link

dim commented Aug 15, 2018

Thanks, much appreciated

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet
None yet

Successfully merging this pull request may close these issues.

None yet

2 participants