Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetch commit offsets in base consumer unless group is None #356

Merged
merged 5 commits into from
Mar 31, 2015
Merged

fetch commit offsets in base consumer unless group is None #356

merged 5 commits into from
Mar 31, 2015

Conversation

dpkp
Copy link
Owner

@dpkp dpkp commented Mar 30, 2015

Fix for Issue #173 (and #347) w/ included test case from @alisaifee

@dpkp dpkp modified the milestone: 0.9.4 Release Mar 30, 2015
@dpkp dpkp changed the title Always fetch commit offsets in base consumer fetch commit offsets in base consumer unless group is None Mar 30, 2015
@dpkp dpkp added the consumer label Mar 30, 2015
if not partitions:
partitions = self.client.get_partition_ids_for_topic(self.topic)

def get_or_init_offset(resp):
for partition in partitions:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to send the offset requests for all partitions at once?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call -- adding a commit for that

alisaifee and others added 4 commits March 30, 2015 15:02
Tweak MP Consumer test to use iterator
a group is configured (but document that group must be None for old
servers). This fixes multiprocessor consumer issue that prevented
access to commit offsets if auto_commit is disabled.
Also refactor fetch_last_known_offsets based on KafkaConsumer
While still setting unknown offsets to 0
dpkp added a commit that referenced this pull request Mar 31, 2015
fetch commit offsets in base consumer unless group is None
@dpkp dpkp merged commit 9fd0811 into dpkp:master Mar 31, 2015
@dpkp dpkp deleted the always_fetch_offsets branch June 10, 2015 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants