New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CommittingSpec: add logging #1007
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Will wait for tests before a merge.
.committableSource( | ||
consumerSettings | ||
.withStopTimeout(100.millis), // this consumer actor needs to stay around to receive the commit | ||
Subscriptions.assignment(new TopicPartition(topic, partition0)) | ||
) | ||
.take(10) | ||
// triggers commit timeout as the actor is terminated | ||
.delay(50.millis) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's good that this resolves the issue, but it looks like it could still arise on systems with slow clocks? We should try repeating this test (i.e. sbt "testOnly thistest -- -DtimesToRepeat=100"
) to see if it fails occasionally.
More long term we should find a way around the stop timeout for happy-path shutdown scenarios.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR fixes the test, not the issue (and added more information to the commit timeout error).
I'm not sure there is a way to fix this, for how long after emitting data should a source live to receive commits? Should the actor stay around, but the stage complete?
Unrelated build error |
CommittingSpec
to keep it from failingIntends to solve #957