Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STORM-2869: Only discard outdated records when adjusting KafkaConsume… #2481

Merged
merged 1 commit into from Dec 29, 2017

Conversation

srdo
Copy link
Contributor

@srdo srdo commented Dec 26, 2017

…r position during commit

See https://issues.apache.org/jira/browse/STORM-2869

I also deleted SingleTopicKafkaSpoutTest, because it is a duplicate of KafkaSpoutSingleTopicTest.

Handling waitingToEmit like this should also fix some edge cases around partition reassignment while there are pending tuples for revoked partitions in waitingToEmit, but I doubt anyone would actually have been affected by that. We never revoke partitions with the built in subscription classes.

Time.advanceTime(KafkaSpoutConfig.DEFAULT_PARTITION_REFRESH_PERIOD_MS + KafkaSpout.TIMER_DELAY_MS);

//The new partition should be discovered and the message should be emitted
spout.nextTuple();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why delete the UT for single topic test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@HeartSaVioR HeartSaVioR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@srdo Thanks for the patch. Could you also provide a patch for 1.x-branch?

@srdo srdo mentioned this pull request Dec 28, 2017
@srdo
Copy link
Contributor Author

srdo commented Dec 28, 2017

Thanks for the reviews. 1.x version here #2484

@asfgit asfgit merged commit df5187e into apache:master Dec 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants