Fix memory leak when reading entry but the connection disconnected. #3528

horizonzy · 2022-10-13T12:55:08Z

Descriptions of the changes in this PR:
Fixes-#3527

eolivelli · 2022-10-13T13:45:51Z

bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBase.java

@@ -145,12 +152,12 @@ protected void sendResponseAndWait(int rc, Object response, OpStatsLogger statsL
        try {
            ChannelFuture future = channel.writeAndFlush(response);
            if (!channel.eventLoop().inEventLoop()) {
-                future.await();


what's the difference in this case ?

await() won't throw the exception out. We'd better catch the exception and log it in debug mode to help debug issues.

eolivelli · 2022-10-13T13:46:39Z

bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java

-        void retain() {
-        }
-
-        void release() {


what about keeping boolean release()
IIUC your problem is about implementing ReferenceCounted

It has conflict with the release() interface in ReferenceCounted

Yeah. I think we can keep boolean release(). Because ReferenceCounted also has the same method.

eolivelli · 2022-10-13T13:47:22Z

bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java

@@ -445,7 +443,7 @@ void recycle() {
    /**
     * A request that reads data.
     */
-    class ReadResponse extends Response {
+    class ReadResponse extends Response implements ReferenceCounted {


why do we need to implement ReferenceCounted ?
can you add a comment in the JavaDoc ?

#3527 will give more context.

hangc0276 · 2022-10-13T14:21:08Z

@horizonzy Great Catch!
There are some tests that failed, please help take a look.
https://github.com/apache/bookkeeper/actions/runs/3242629439/jobs/5316169295

zymap · 2022-10-14T01:23:25Z

bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBase.java

-            channel.writeAndFlush(response, channel.voidPromise());
+            ChannelPromise promise = channel.newPromise().addListener(future -> {
+                if (!future.isSuccess()) {
+                    logger.debug("Netty channel write exception. ", future.cause());


Why not log it as an error?

When this error occurs, it means that the client has already close. Log error is unnecessary, the other log will show the connection already close.

I think this need to print the error
logger.error("Netty channel write exception. ", future.cause());

bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBase.java

zymap · 2022-10-14T01:29:52Z

bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java

-        void retain() {
-        }
-
-        void release() {


Yeah. I think we can keep boolean release(). Because ReferenceCounted also has the same method.

michaeljmarshall

LGTM

hangc0276 · 2022-10-17T03:49:22Z

rerun failure checks

wolfstudy · 2022-10-17T11:29:25Z

ping @eolivelli PTAL again, thanks

eolivelli

LGTM

eolivelli · 2022-10-17T14:03:10Z

@StevenLuMT @zymap do you have other comments ?

dlg99

Nicely done.

zymap

LGTM

StevenLuMT

LGTM

…3528) * Fix direct memory leak problem (cherry picked from commit 30bdedc)

…pache#3528) * Fix direct memory leak problem (cherry picked from commit 30bdedc)

…pache#3528) * Fix direct memory leak problem (cherry picked from commit 30bdedc) (cherry picked from commit 9329e3c)

…ected. (apache#3528)" This reverts commit cef988f.

lhotari · 2023-01-20T11:50:41Z

bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBase.java

+            ChannelPromise promise = channel.newPromise().addListener(future -> {
+                if (!future.isSuccess()) {
+                    logger.debug("Netty channel write exception. ", future.cause());
+                }
+            });


This will cause unnecessary object creation. If the only purpose is to log exceptions, channel.voidPromise() already takes care of that.

Thanks for reminder.

It has been fixed here: #3733

…pache#3528) * Fix direct memory leak problem (cherry picked from commit 30bdedc) (cherry picked from commit 9329e3c)

…pache#3528) * Fix direct memory leak problem

horizonzy added 3 commits October 13, 2022 18:07

Fix direct memory leak problem

28a4b23

Fix direct memory leak problem.

1e397d2

Fix checkstyle.

ee23d2e

eolivelli requested changes Oct 13, 2022

View reviewed changes

hangc0276 requested review from zymap, merlimat, dlg99 and ivankelly October 13, 2022 14:18

hangc0276 assigned horizonzy Oct 13, 2022

hangc0276 added release/4.14.6 release/4.15.3 labels Oct 13, 2022

hangc0276 added this to the 4.16.0 milestone Oct 13, 2022

horizonzy added 3 commits October 13, 2022 23:33

Fix the ci.

5db2f6a

revert code

c689ade

fix checkstyle.

2bf5d7f

zymap added type/bug area/bookie labels Oct 14, 2022

zymap reviewed Oct 14, 2022

View reviewed changes

gaozhangmin approved these changes Oct 14, 2022

View reviewed changes

merlimat approved these changes Oct 14, 2022

View reviewed changes

make Response implements boolean release()

ba307ab

hangc0276 approved these changes Oct 14, 2022

View reviewed changes

michaeljmarshall approved these changes Oct 14, 2022

View reviewed changes

shoothzj approved these changes Oct 15, 2022

View reviewed changes

eolivelli approved these changes Oct 17, 2022

View reviewed changes

dlg99 approved these changes Oct 17, 2022

View reviewed changes

zymap approved these changes Oct 18, 2022

View reviewed changes

StevenLuMT approved these changes Oct 18, 2022

View reviewed changes

StevenLuMT merged commit 30bdedc into apache:master Oct 18, 2022

zymap pushed a commit that referenced this pull request Oct 26, 2022

Fix memory leak when reading entry but the connection disconnected. (#…

3a95424

…3528) * Fix direct memory leak problem (cherry picked from commit 30bdedc)

zymap added the cherry-picked/branch-4.15 label Oct 26, 2022

hangc0276 pushed a commit to hangc0276/bookkeeper that referenced this pull request Nov 7, 2022

Fix memory leak when reading entry but the connection disconnected. (a…

612dff6

…pache#3528) * Fix direct memory leak problem (cherry picked from commit 30bdedc)

hangc0276 pushed a commit to hangc0276/bookkeeper that referenced this pull request Nov 7, 2022

Fix memory leak when reading entry but the connection disconnected. (a…

9329e3c

…pache#3528) * Fix direct memory leak problem (cherry picked from commit 30bdedc)

hangc0276 added the cherry-picked/branch-4.14 label Nov 7, 2022

lordcheng10 mentioned this pull request Nov 15, 2022

[bookie] Fix memory leak for PacketProcessorBaseV3 #3644

Closed

nicoloboschi added a commit to datastax/bookkeeper that referenced this pull request Jan 20, 2023

Revert "Fix memory leak when reading entry but the connection disconn…

c7e834f

…ected. (apache#3528)" This reverts commit cef988f.

lhotari reviewed Jan 20, 2023

View reviewed changes

MMirelli mentioned this pull request Jan 20, 2023

Pulsar broker cannot start due to bookies timing out on putting schema #3751

Closed

horizonzy mentioned this pull request Feb 13, 2023

Add test to cover ReferenceCountUtil.touch in BookieProtocol. #3777

Closed

Ghatage pushed a commit to sijie/bookkeeper that referenced this pull request Jul 12, 2024

Fix memory leak when reading entry but the connection disconnected. (a…

7a9d753

…pache#3528) * Fix direct memory leak problem

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix memory leak when reading entry but the connection disconnected. #3528

Fix memory leak when reading entry but the connection disconnected. #3528

horizonzy commented Oct 13, 2022

eolivelli Oct 13, 2022

hangc0276 Oct 13, 2022

eolivelli Oct 13, 2022

hangc0276 Oct 13, 2022

zymap Oct 14, 2022

horizonzy Oct 14, 2022

eolivelli Oct 13, 2022

hangc0276 Oct 13, 2022

hangc0276 commented Oct 13, 2022

zymap Oct 14, 2022

horizonzy Oct 14, 2022

StevenLuMT Oct 15, 2022

zymap Oct 14, 2022

michaeljmarshall left a comment

hangc0276 commented Oct 17, 2022

wolfstudy commented Oct 17, 2022

eolivelli left a comment

eolivelli commented Oct 17, 2022

dlg99 left a comment

zymap left a comment

StevenLuMT left a comment

lhotari Jan 20, 2023

horizonzy Jan 20, 2023

nicoloboschi Jan 20, 2023

Fix memory leak when reading entry but the connection disconnected. #3528

Fix memory leak when reading entry but the connection disconnected. #3528

Conversation

horizonzy commented Oct 13, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hangc0276 commented Oct 13, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaeljmarshall left a comment

Choose a reason for hiding this comment

hangc0276 commented Oct 17, 2022

wolfstudy commented Oct 17, 2022

eolivelli left a comment

Choose a reason for hiding this comment

eolivelli commented Oct 17, 2022

dlg99 left a comment

Choose a reason for hiding this comment

zymap left a comment

Choose a reason for hiding this comment

StevenLuMT left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment