Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
KAFKA-5998: fix checkpointableOffsets handling #7030
Committer Checklist (excluded from commit message)
vvcephei left a comment
This PR should fix the long-running KAFKA-5998 bug.
The change I'm proposing is bigger in scope than the actual fix, though, because I wanted to take steps to prevent a similar bug from cropping up in the same code in the future. My theory is that the bug had an easy time hiding in this code because the handling of checkpointable offsets was so complex. I'm hoping that by reducing the mutable scope and also tightening up the invariants around the checkpointable offsets, we will have an easier time maintaining this module.
Let me know what you think!
cadonna left a comment
Thanks for the PR @vvcephei and congratulations to you and @pkleindl for finding and fixing this bug. I left a couple of comments. I had hard time to find the code that actually fixes the bug (and I am still not sure if I found it). Could you please add some specific comments about the fix, next time, since this fix is not that trivial? I am also wondering if you could have divided this PR in two: one for the fix itself and one for the repair of old corrupted checkpoints. IMO, it would have made reviewing the PRs easier.
Thanks for the review, @cadonna !
I'm sorry that my earlier call-out of the actual bugfix got marked "outdated" at some point, so I guess it doesn't show up in the diff anymore. I didn't notice when that happened, or I would have re-marked it.
Regarding splitting up the PRs, I do agree with you. It would have been nice to get a smaller fix in, and then tackled the refactoring separately.
If I can make one excuse for myself, it would be that in this case, it wasn't clear to me that the fix was good enough because the scope of
In retrospect, though, I could have submitted the refactor first, and then followed up with the bugfix. It just didn't occur to me, for whatever reason.
In any case, thanks for wading through the code review! I think I addressed all your comments.