Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ResourceLeakDetector also emit recent access records for IllegalReferenceCountException #9892

Open
wants to merge 5 commits into
base: 4.1
Choose a base branch
from

Conversation

carryxyh
Copy link
Contributor

Motivation:
Let IllegalReferenceCountException report the access log when over-released.

Modification:

  • Add AdvancedResourceLeak, and keep the record when close.
  • Print exception information during release or retain.

Result:
Fixes #9229

@netty-bot
Copy link

Can one of the admins verify this patch?

}

@Override
public ByteBuf retain(int increment) {
leak.record();
return super.retain(increment);
try {
return super.retain(increment);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like other operation(like getBytes) also need to catch IllegalReferenceCountException.

private static final long serialVersionUID = 1374377399979428484L;

public TrackedIllegalReferenceCountException(String message, String accessRecords) {
super(message + StringUtil.NEWLINE + accessRecords);
Copy link
Contributor Author

@carryxyh carryxyh Dec 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since Record actually a throwable. Should we let Record and TrackedIllegalReferenceCountException constitute a parent-child relationship? like:

TrackedIllegalReferenceCountException.cause = head
head.cause = next
...
next.cause = tail

This means we should let Record not only a private class.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I can follow here. Can you explain what exactly this would buy us ?

Copy link
Contributor Author

@carryxyh carryxyh Dec 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course.
I made a simple test to illustrate:

public static void main(String[] args) {

    // here is three record.
    //
    // head.next => second.
    // second.next => tail.
    Record tail = new Record();
    Record second = getBytes(tail);
    Record head = getInt(second);

    head.initCause(second);
    second.initCause(tail);

    NullPointerException t = new NullPointerException();

    t.initCause(head);

    throw t;
}

// mock two record.

private static Record getBytes() {
    return new Record();
}

private static Record getInt() {
    return new Record();
}

Running the above example, we will get exception information:

Exception in thread "main" java.lang.NullPointerException
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:676)
Caused by: 
	at io.netty.util.ResourceLeakDetector.getInt(ResourceLeakDetector.java:690)
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:671)
Caused by: 
	at io.netty.util.ResourceLeakDetector.getBytes(ResourceLeakDetector.java:686)
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:670)
Caused by: 
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:669)

If I remove t.initCause (head), it only produces:

Exception in thread "main" java.lang.NullPointerException
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:676)

The NullPointerException in the example is our TrackedIllegalReferenceCountException.

Note that Record itself is a Throwable. So we can use initCause to associate Record with TrackedIllegalReferenceCountException, so that for users, they can directly obtain Record information from the stack.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I continue to enhance this example, I make a small modification in the Record # toString method:

image

Running the above example also, I will get:

Exception in thread "main" java.lang.NullPointerException
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:679)
Caused by: record info
	at io.netty.util.ResourceLeakDetector.getInt(ResourceLeakDetector.java:693)
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:674)
Caused by: record info
	at io.netty.util.ResourceLeakDetector.getBytes(ResourceLeakDetector.java:689)
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:673)
Caused by: record info
	at io.netty.util.ResourceLeakDetector.main(ResourceLeakDetector.java:672)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since TrackedIllegalReferenceCountException is an exception, and many times we will not catch it, so when this exception is thrown, it may be a good way to intuitively display the Record information displayed in the stack.

}

@Override
protected void finalize() throws Throwable {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need release head when AdvancedResourceLeak will be gced..?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this finalize should not be here.

@normanmaurer
Copy link
Member

@netty-bot test this please

@carryxyh
Copy link
Contributor Author

@normanmaurer
Could you please check my comment? To complete the full pull request, I need more suggestions..

@normanmaurer
Copy link
Member

@carryxyh I will try soon.

// means leak has been closed and there is no accessRecord.
return e;
}
return new TrackedIllegalReferenceCountException(e.getMessage(), accessRecord);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we wrap the original e ?

@@ -0,0 +1,30 @@
/*
* Copyright 2012 The Netty Project
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2019

private static final long serialVersionUID = 1374377399979428484L;

public TrackedIllegalReferenceCountException(String message, String accessRecords) {
super(message + StringUtil.NEWLINE + accessRecords);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I can follow here. Can you explain what exactly this would buy us ?

/**
* A special {@link IllegalReferenceCountException} with the ability to track access records
*/
public class TrackedIllegalReferenceCountException extends IllegalReferenceCountException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we make this package-private for now and final ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this exception will be used in AdvancedLeakAwareByteBuf, this can't be a package-private class..

}

@Override
protected void finalize() throws Throwable {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this finalize should not be here.

@carryxyh
Copy link
Contributor Author

Fixed.. Pls check..

@normanmaurer
Copy link
Member

@carl-mastrangelo WDYT ?

@gerdriesselmann
Copy link
Contributor

What's the state of affairs regarding this patch? We still encounter #9882, which is difficult to fix without the information this patch will provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make ResourceLeakDetector also emit recent access records for IllegalReferenceCountException
5 participants