-
Notifications
You must be signed in to change notification settings - Fork 13.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KAFKA-16663: cancel write timeout TimerTask on successful event completion #15902
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jeffkbkim Thanks for the fix. I left one comment.
@@ -1083,6 +1094,11 @@ public void run() { | |||
public void complete(Throwable exception) { | |||
if (exception == null) { | |||
future.complete(null); | |||
|
|||
if (writeTimeoutTask != null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that we should cancel it whenever the event is completed, regardless of whether it is successful or not. In both cases, it is completed so the timer has no reason to continue. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i originally had it only when it completes successfully because TimerTask#cancel requires synchronization on SystemTimer. I guess we will need to get a lock on it anyways when it expires anyways.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks.
…etion (apache#15902) Write events create and add a TimerTask to schedule the timeout operation. The issue is that we pile up the number of timer tasks which are essentially no-ops if replication was successful. They stay in memory for 15 seconds (default write timeout) and as the rate of write increases, the impact on memory usage increases. Instead, cancel the corresponding write timeout task when the write event is committed to the log. This also applies to complete transaction events. Reviewers: David Jacot <djacot@confluent.io>
…etion (apache#15902) Write events create and add a TimerTask to schedule the timeout operation. The issue is that we pile up the number of timer tasks which are essentially no-ops if replication was successful. They stay in memory for 15 seconds (default write timeout) and as the rate of write increases, the impact on memory usage increases. Instead, cancel the corresponding write timeout task when the write event is committed to the log. This also applies to complete transaction events. Reviewers: David Jacot <djacot@confluent.io>
write events create and add a TimerTask to schedule the timeout operation. The issue is that we pile up the number of timer tasks which are essentially no-ops if replication was successful. They stay in memory for 15 seconds (default write timeout) and as the rate of write increases, the impact on memory usage increases.
Instead, cancel the corresponding write timeout task when the write event is committed to the log. This also applies to complete transaction events.
Committer Checklist (excluded from commit message)