TestFX 4.0.7-alpha introduced larger memory use

[mvsoder]

Starting with release 4.0.7-alpha, running TestFX with monocle uses more memory than before and has started causing OutOfMemoryErrors. Increasing the max JVM memory has mitigated the issue but over 1G of memory was used on fewer than 20 tests. This has been hard to track down because support for Java 9 was also introduced in this time period. Testing was done using Oracle Java 1.8.0_144 on Ubuntu Server 17.04. The higher memory use continues in TestFX 4.0.8-alpha.

[brcolow]

Can you git bisect to a particular commit? That would be helpful.

[mvsoder]

I have attempted to reproduce this problem several times but have not been able to. Even running with Monocle I cannot reproduce the problem. Please close the issue.

…

On Fri, Nov 10, 2017 at 11:35 PM, Michael Ennen ***@***.***> wrote: Can you git bisect to a particular commit? That would be helpful. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#448 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADLXtB89my2KHU2k7KCPvdj-MZzoKxRgks5s1UBAgaJpZM4QSNjX> .

[brcolow]

Good to hear. Thanks for getting back to me!

[mvsoder]

I think I have reproduced this. It did not start with 4.0.7, but I think it may still have to do with TestFX. I can reproduce the problem when running a unit test using TestFX but not when running a simple application. It think it has to do with an application that creates and shows more stages than the primary stage. I have SSCCE if it helps.

[brcolow]

Sure an SSCCE can only help :).

[mvsoder]

As simple an SSCCE I could create is attached.
fxleakdemo.zip

[brcolow]

What is the purpose of creating a new Stage and not using the one that is passed in via the start method? Also, what happens if you call initOwner(stage) on the new Stage, where the stage argument to initOwner is the one passed via start?

[mvsoder]

Both valid questions. My application allows the user to open more Stages as workspaces. The original stage is used as a splash screen window. I had not considered calling initOwner on the new stages as they were expected to "stand on their own".

[brcolow]

I'm just curious if calling initOwner changes the leak at all. I am not really sure how we could fix this on the TestFX side, practically speaking. How can we check if Stage constructor was called, but not used, in a start method? Unless I am thinking about this incorrectly. Are you thinking we could query JavaFX to tell us if any stages were constructed and then dispose of them?

[mvsoder]

Note in my example that the new stage is used, it simply replaces the original. It should be in the list returned by Window.getWindows(). Furthermore, I changed the code so that the new stage called initOwner with no change.

[brcolow]

Ah, sorry, I misunderstood one of the comments. Will think about how to proceed, PR welcome (I will try and take a stab at fixing this within the next couple days).

[brcolow]

A couple things:

1.) Calling Runtime.getRuntime().gc(); does not guarantee that garbage collection will be performed. Adding -Xlog:gc* to the JDK_JAVA_OPTIONS environment variable shows that indeed garbage collection is not performed after any of the 10 tests.

2.) We can not close all windows/stages after each and every test, because a TestFX user may want to keep some/all stages open in-between tests. Thus our only option is to close the windows after each test class.

3.) Even if we could close all windows/stages after each test, all we can do using the JavaFX API is to call hide() on each Window. This will not make the stage be garbage collected. This is what FxToolkit.cleanupStages() does, but as you note even when this is called it doesn't make a difference.

4.) I believe the problem, if there is one, is that there is a reference somewhere to the stage passed to start (even though Java GC is not reference counted, it is more complex than that but just for the sake of easily discussing this issue) being held when it should be gone. I'd like to know where that is, but tracking it down is somewhat difficult. However there also may not actually be a problem because I have not been able to trigger a garbage collection with the sample.

tl;dr: Runtime.getRuntime().gc() is not guaranteed to trigger a GC, and using a JVM option (-Xlog:gc*) to monitor GC shows that none are performed during the SSCCE so it's impossible to reason about if there is a memory leak or not.

This is all to the best of my understanding, and I could easily be mistaken about some things. I am not trying to make this difficult, just don't know how to proceed and not sure that we have demonstrated there actually is a leak, yet.

[mvsoder]

All good points.:

4. I agree with your assessment that there appears to be something holding on to the Stage references. My assumption is that the references are to the new Stages because they have the large image in them, the original stage does not. It would take a lot of original stages building up to run out of memory. The odd part is, I can't figure out what might be holding on to those new Stage references, especially since the object is created locally.

5. What I find interesting about the original stage is that if I use it, the tests work just fine. Memory is cleaned up, not errors, etc. Only when I create the second stage does the test start failing.

1.a) I ran my tests again with -Xlog:gc* enabled and unless I'm interpreting incorrectly I do see the garbage collector working. Here is a small exerpt:
[1.515s][info][gc,start ] GC(1) Pause Full (System.gc())
[1.516s][info][gc,phases,start] GC(1) Phase 1: Mark live objects
Eventually I get the following right before the OutOfMemoryError:
[3.237s][info][gc ] GC(19) Pause Full (Allocation Failure) 246M->245M(256M) 25.789ms

1.b) You are correct that running Runtime.getRuntime().gc() does not guarantee garbage collection. The intent of using it was to give it a change to collect. According to the Runtime.gc() docs, "When control returns from the method call, the virtual machine has made its best effort to recycle all discarded objects." That's all I was hoping for from this method call.

1.c) More interesting is the OutOfMemoryError that I get around runTest4. According to the OutOfMemoryError documentation, "the Java Virtual Machine cannot allocate an object because it is out of memory, and no more memory could be made available by the garbage collector." This is a bit more firm about the fact the garbage collector did try to make memory available. There is also this document, https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html, which pretty much details out the same concept.

I'll keep plinking away at this and see if I can figure out what's holding on to the object references. It may still not be a TestFX issue, just that I can only reproduce it there at the moment. Best regards.

[brcolow]

Okay, so there is definitely a leak. Could it be JavaFX itself?

[mvsoder]

It could be JavaFX itself but I have yet to get it to behave incorrectly using the FxFrameApp. As long as I close the stages it always reclaims the memory, just as we expect. If I don't close the stages, it quickly runs out of memory, just as we expect. I might be forced to dust of my profiling skills for this. :-)

[mvsoder]

I found another interesting symptom. If the application does not hold on to a reference to the new stage, then there is not a leak. If the application does hold a reference to the original stage there is also no leak. But if the application does hold a reference to the new stage, then there is a leak. However, using a SoftReference or a WeakReference to the new stage does not cause a leak. The implication is that there is a reference to each instance of the application and that, in turn, holds a reference to the stage that is causing the leak.

[mvsoder]

Another clue. Holding on to references to the applications or the stages from the unit test class does not cause the leak, but holding on to references to the scenes does cause the leak. I'm now looking for something that holds on to scene references.

[mvsoder]

After many more months of observation I believe the memory issues are in monocle and not TestFX. I believe this issue should be closed now.