Direct Memory leak in RatPack 1.3.3

[ronhitchens]

From a post on the Slack channel:

= - = - = - =
Hey guys. I asked about Direct Memory leaks in RP 1.3.3 a few days ago. Well I finally got around to testing it with a JVM monitor and it is indeed leaking. Big time. I have 2GB of DM configured and I can watch it all leak away in just a few minutes. I can post graphs if anyone wants to see them.

I updated to 1.4.2 and the leak is gone. Nice and stable. Steady around 500MB of DM when pushed hard. I don’t know where the leak is, but I have empirical evidence that the latest RatPack release fixes it so I’m moving on. But you may want to inform your users that 1.3.3 has a serious DM leak. We do a lot of HTTPClient calls, sending and receiving large-ish PDFs, which probably make it leak much faster than for most people.

It could probably be reproduced by using HTTPClient to fetch a 15-20MB PDF from any web server repeatedly. I used the siege tool to fire requests, 10 or 20 concurrently, while watching it with VisualVM. Note: You will need to set the system property -Dio.netty.maxDirectMemory=0 to see memory usage in a monitor. Otherwise Netty does something funky with the direct memory pool that prevents the MBean from seeing the actual pool state.
= - = - = - =

Attaching two graphs from VisualVM, one using 1.3.3, the other using 1.4.2. These were watching runs by the tool "siege" which was requesting the same URL repeatedly, with 20 running concurrently. Each request finds a PDF on the local filesystem (for testing, normally it would come from Amazon S3) then sends the PDF using HttpClient to another service which stamps it and sends it back. The stamped PDF is then returned as the body of the final response. The PDF is about 15MB in size.

[johnrengelman]

Seeing this type of behavior with a proxy type app as well.
Using HttpClient and doing a response.forwardTo(response)

I haven't been able to find any Ratpack code changes between 1.3.3 and 1.4.2 that stand out as candidates for memory leak but the biggest difference is the changes in the HttpClient code to support pooling that came in 1.4.

[danhyun]

Keep in mind we did upgrade Netty from 4.1.0.CR7 in ratpack 1.3.3 to 4.1.5.Final in ratpack 1.4.2

the changelist for that can be viewed here netty/netty@netty-4.1.0.CR7...netty-4.1.4.Final

[ldaley]

I made some major changes in the StreamedResponse area for 1.4. It's reasonably likely I fixed the leak in that.

[johnrengelman]

Cool.