Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Fix transaction offsets for transactional producer #742
Pull Request Checklist
Use maximal offset when committing transaction in
This fix ensures that we only increase the offsets stored in
See #539 for some relevant context.
Are there any relevant issues / PRs / mailing lists discussions?
referenced this pull request
Mar 8, 2019
@ennru I cannot easily force callbacks to return in specific other.
However, the probability of the one with the highest order to return last is
@szymonm I assume this question was directed to me?
I think that's probably the best we can do right now without modifying the
You're right @seglo.
We need to have at least two consumers to test that offsets are stored in Kafka correctly (more if we want the test to be more reliable in finding errors).
@szymonm I'm not sure I get it.
I'm attempting to understand what's happening in your test provided with this PR. It seems that it sequentially runs transactional streams for 5 batches of numbers. i.e.)
The source test data used in each transaction is always coming from a single partition, but is produced to multiple partitions using the round robin partioning strategy because a key isn't defined in the
That's right, @seglo .
One thing I would add is that we don't impose any order in which the streams process data (the