Netty 4/5 LACK OF deterministic buffer leak detection that can be used in unit tests

[codekrolik]

I read a lot about buffer leak detection in Netty4 and to me it looks like there is no deterministic way to detect such leaks in unit tests.

However, the importance of such feature for unit testing is so great that it feels very wrong that there are no clear guidelines on how to do it.

Moreover, most of the sources including the original document
http://netty.io/wiki/reference-counted-objects.html
make things amazingly confusing by giving vague hints like:

"The following output shows a leak from our unit test (XmlFrameDecoderTest.testDecodeWithXml()):"

which sounds like there exists a way to deterministically detect leaks in buffer allocators in unit tests, while in fact there is no such thing, as Trustin Lee himself notes in his answers (link below).

No wonder this was reposted dozens of times by various sources that have no idea and just copy-paste words without testing.

_) Trustin Lee suggested running an application for 30 seconds under some busy workload in the following topic.
http://stackoverflow.com/questions/28822632/netty-4-5-does-not-actually-detect-resource-leak-of-bytebuf
_However that doesn't trigger detection or any output from ResourceLeakDetector for me.*
Also, using this approach is a horrible practice for unit tests.

*) I also tried the GC trick suggested in the following topic
http://stackoverflow.com/questions/1481178/how-to-force-garbage-collection-in-java
but that doesn't make any difference either.

GC is so unpredictable that it's very tough to imagine how ResourceLeakDetector can be leveraged to create clean and thorough buffer leak unit tests.

*) Another way is to test refCnt for every single ByteBuf that was created while a test was running.
But sometimes it's impossible to get hold of every such reference, because an interface might declare String as an input parameter and then its implementation would create and release a ByteBuf instance internally, and that reference will be unreachable for unit test, however if a release didn't happen it would produce a leak with no chance to detect in unit tests.

*) I also couldn't find an easy way to get a list of all existing buffers from the allocator, otherwise it would be possible to just check refCnt for every single one of them.

---

So far to me it seems like unit tests are useless for this purpose, unless you run a full-scale server for extended periods of time in your unit tests (and this by the way also doesn't guarantee you anything, just increases your chances in theory).
As far as I can see, there is no good reason for such limitations to testing to exist. Providing buffer allocations and release count from allocator for unit tests can't be hard.

[normanmaurer]

@codekrolik I guess we could add some custom @Rule for junit that can be used. Need to think a bit more about this tho.

[codekrolik]

So I managed to make unit tests work for me.

The mechanism is like the following:

1. Create a PooledBufferAllocator with disabled cache, as was suggested in Netty 4 PooledByteBufAllocator returns incorrect information about allocations/deallocations #5275

PooledByteBufAllocator alloc = new PooledByteBufAllocator(true, 1, 1, 8192, 11, 0, 0, 0);

2. Make sure all the Bootstraps use this allocator

a. Client

Bootstrap clientBootstrap = new Bootstrap();
clientBootstrap.option(ChannelOption.ALLOCATOR, alloc);

b. Server

ServerBootstrap serverBootstrap = new ServerBootstrap();
serverBootstrap.option(ChannelOption.ALLOCATOR, alloc)
    .childOption(ChannelOption.ALLOCATOR, alloc);

3. After the test is done, check buffer leaks for direct and heap buffers

assertEquals(0, getActiveDirectBuffers(alloc));
assertEquals(0, getActiveHeapBuffers(alloc));

int getActiveDirectBuffers(PooledByteBufAllocator alloc) {
    int directActive = 0, directAlloc = 0, directDealloc = 0;
    for (PoolArenaMetric arena : alloc.directArenas()) {
        directActive += arena.numActiveAllocations();
        directAlloc += arena.numAllocations();
        directDealloc += arena.numDeallocations();
    }
    System.out.println("directActive " + directActive + " directAlloc " + directAlloc + " directDealloc " + directDealloc);
    return directActive;
}

int getActiveHeapBuffers(PooledByteBufAllocator alloc) {
    int heapActive = 0, heapAlloc = 0, heapDealloc = 0;
    for (PoolArenaMetric arena : alloc.heapArenas()) {
        heapActive += arena.numActiveAllocations();
        heapAlloc += arena.numAllocations();
        heapDealloc += arena.numDeallocations();
    }
    System.out.println("heapActive " + heapActive + " heapAlloc " + heapAlloc + " heapDealloc " + heapDealloc);
    return heapActive;
}