[ML] Release OpAddEntry.data when entry is copied and discarded #10773
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
merlimat
approved these changes
Jun 1, 2021
sijie
reviewed
Jun 1, 2021
| @@ -1504,6 +1504,8 @@ public synchronized void updateLedgersIdsComplete(Stat stat) { | |||
| if (existsOp.ledger != null) { | |||
| existsOp.close(); | |||
| existsOp = OpAddEntry.create(existsOp.ml, existsOp.data, existsOp.getNumberOfMessages(), existsOp.callback, existsOp.ctx); | |||
| // release the extra retain | |||
| ReferenceCountUtil.release(existsOp.data); | |||
There was a problem hiding this comment.
Just one question, doesn't existsOp.close release the reference? Or can you clarify (potentially comment in the code) why do we need to release extra retain here?
There was a problem hiding this comment.
Just one question, doesn't existsOp.close release the reference?
No it doesn't.
why do we need to release extra retain here?
OpAddEntry.create will call .retain , that's the extra retain. I already tried to clarify that in the comment in the code. Does it need more clarification?
There was a problem hiding this comment.
LGTM, just add more context here. We can safe to release the buffer here because we closed the old OpAddEntry, if the OpAddEntry in the closed state, even if the addComplete been called, will not use the buffer again.
if (!STATE_UPDATER.compareAndSet(OpAddEntry.this, State.INITIATED, State.COMPLETED)) {
log.warn("[{}] The add op is terminal legacy callback for entry {}-{} adding.", ml.getName(), lh.getId(), entryId);
OpAddEntry.this.recycle();
return;
}
|
Linking to #6054 for tracking |
codelipenghui
approved these changes
Jun 2, 2021
| @@ -1504,6 +1504,8 @@ public synchronized void updateLedgersIdsComplete(Stat stat) { | |||
| if (existsOp.ledger != null) { | |||
| existsOp.close(); | |||
| existsOp = OpAddEntry.create(existsOp.ml, existsOp.data, existsOp.getNumberOfMessages(), existsOp.callback, existsOp.ctx); | |||
| // release the extra retain | |||
| ReferenceCountUtil.release(existsOp.data); | |||
There was a problem hiding this comment.
LGTM, just add more context here. We can safe to release the buffer here because we closed the old OpAddEntry, if the OpAddEntry in the closed state, even if the addComplete been called, will not use the buffer again.
if (!STATE_UPDATER.compareAndSet(OpAddEntry.this, State.INITIATED, State.COMPLETED)) {
log.warn("[{}] The add op is terminal legacy callback for entry {}-{} adding.", ml.getName(), lh.getId(), entryId);
OpAddEntry.this.recycle();
return;
}
wangjialing218
pushed a commit
to wangjialing218/pulsar
that referenced
this pull request
Jun 2, 2021
) Fixes apache#10738 ### Motivation There's a ByteBuf leak that happens in ledger rollover. The repro case in apache#10738 reproduces some Netty ByteBuf leaks that are detected by the Netty Leak detector. ### Modifications Release `OpAddEntry.data` when the original entry is copied and discarded ### Additional Context This PR was split out of PR apache#10755 which was closed without merging.
lhotari
added a commit
that referenced
this pull request
Jun 2, 2021
Fixes #10738 There's a ByteBuf leak that happens in ledger rollover. The repro case in #10738 reproduces some Netty ByteBuf leaks that are detected by the Netty Leak detector. Release `OpAddEntry.data` when the original entry is copied and discarded This PR was split out of PR #10755 which was closed without merging.
lhotari
added a commit
that referenced
this pull request
Jun 2, 2021
Fixes #10738 There's a ByteBuf leak that happens in ledger rollover. The repro case in #10738 reproduces some Netty ByteBuf leaks that are detected by the Netty Leak detector. Release `OpAddEntry.data` when the original entry is copied and discarded This PR was split out of PR #10755 which was closed without merging.
eolivelli
pushed a commit
to datastax/pulsar
that referenced
this pull request
Jun 7, 2021
) Fixes apache#10738 There's a ByteBuf leak that happens in ledger rollover. The repro case in apache#10738 reproduces some Netty ByteBuf leaks that are detected by the Netty Leak detector. Release `OpAddEntry.data` when the original entry is copied and discarded This PR was split out of PR apache#10755 which was closed without merging. (cherry picked from commit 121305f)
yangl
pushed a commit
to yangl/pulsar
that referenced
this pull request
Jun 23, 2021
) Fixes apache#10738 ### Motivation There's a ByteBuf leak that happens in ledger rollover. The repro case in apache#10738 reproduces some Netty ByteBuf leaks that are detected by the Netty Leak detector. ### Modifications Release `OpAddEntry.data` when the original entry is copied and discarded ### Additional Context This PR was split out of PR apache#10755 which was closed without merging.
bharanic-dev
pushed a commit
to bharanic-dev/pulsar
that referenced
this pull request
Mar 18, 2022
) Fixes apache#10738 ### Motivation There's a ByteBuf leak that happens in ledger rollover. The repro case in apache#10738 reproduces some Netty ByteBuf leaks that are detected by the Netty Leak detector. ### Modifications Release `OpAddEntry.data` when the original entry is copied and discarded ### Additional Context This PR was split out of PR apache#10755 which was closed without merging.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #10738
Motivation
There's a ByteBuf leak that happens in ledger rollover. The repro case in #10738 reproduces some Netty ByteBuf leaks that are detected by the Netty Leak detector.
Modifications
Release
OpAddEntry.datawhen the original entry is copied and discardedAdditional Context
This PR was split out of PR #10755 which was closed without merging.