Memory leak in mockito-inline calling method on mock with at least a mock as parameter

[ttanxu]

Summary

We found a memory leak with mockito-inline. A short example is shown as below. A more detailed example can be found at the end.

class BigClass {
  void accept(SmallClass small) {}
}

class SmallClass {
  void accept(BigClass big) {}
}

void leak() {
  BigClass a = Mockito.mock(BigClass.class);
  SmallClass b = Mockito.mock(SmallClass.class);

  // Remove any one of the 2 lines below will stop leaking
  a.accept(b);
  b.accept(a);
}

Analysis

Looking at the memory dump. Mock of BigClass and SmallClass are held as a weak ref in a map of type WeakConcurrentMap$WithInlinedExpunction used to map the mock and its invocation handler. When the mock can be reclaimed by GC the map will remove the record from it.

In the case with memory leak, the mock instance of BigClass was held as a strong reference by rawArguments and arguments in InvocationMatcher, as invocation in InterceptedInvocation, as invocationForStubbing in InvocationContainerImpl, as invocationContainer in MockHandlerImpl. That eventually leads to a value in the WeakConcurrentMap for SmallClass. Similar thing happens to the mock instance of SmallClass. That creates a ring of reference and no mock can be reclaimed by GC because they are all referenced transitively by a value in the map.

Similar things can also happen for stubbed methods, saved in stubbed in InvocationContainerImpl.

Subclass mock makers don't suffer from it because there is no map from mock to handler -- it's just a strong reference. GC can handle non-accessible rings well, but GC doesn't know the mock map purging semantic in inline mock makers.

Potential Solution

Unlike #1533 where converting spiedInstance to a weak reference may be an acceptable solution, we can't convert arguments in stubbing method calls into weak references because there are stubbing calls with an object (or a mock) that doesn't have strong ref anywhere else than arguments in Mockito.

Therefore I failed to see a solution that's transparent to callers. The possible solution below is the one that I think has the least change.

The possible solution is we can somehow reset the mock when their lives end, which clears the stubbing records. Maybe we can tie their lives to a MockitoSession. We can track all mocks created after a session is created, and reset them when the session is finished. Now we only have events for mock creation in the same thread, we may need to expand that to include other threads.

We may be able to set spiedInstance to null when their lives end, which should also be able to fix #1532 and #1533. That would be a solution with least risk.

Of course any solution that's transparent to callers is still more desirable.

A Detailed Example

See GitHub project MockitoMethodCallMemLeak. One can open it in IntelliJ and the run configuration is already configured (with mem dump at OOM and 4M java heap size).

[TimvdLippe]

Thanks for the detailed explanation!

We may be able to set spiedInstance to null when their lives end, which should also be able to fix #1532 and #1533. That would be a solution with least risk.

Sounds like the best plan forward. Do you mind opening a PR with regression test to get that fixed?

[ttanxu]

It seems to be rather hard to convert all listeners to listen to events from all threads due to current test setup.

However it's pretty important to capture mocks from other threads to make this solution work, because JUnit has failOnTimeout impl which basically runs each individual test case in its own thread. That means we need start a session in each test case, rather than in setUp and tearDown, which is rather inconvenient.

I'll add a global listener that captures all mock events from all threads, but disable the feature unless caller explicitly enables in session builders.

It would be a hard refactor to make ThreadsRunAllTestsHalfManualTest pass with this.

[ttanxu]

Actually we probably shouldn't refactor those tests, since they probably represent part of current API. But it should be just OK to stick to the plan in my previous comment.

[joshallenit]

Thanks! I added documentation about this as it was having a big impact on a project and was hard to find. https://github.com/mockito/mockito/wiki/What's-new-in-Mockito-2#mockito-2250

[marius-bardan]

we just updated our testing dependencies to latest version and ran into this OOM.
how is adding an '@after'-annotated function to 3000+ tests practical, of which some already have their own implementation '@after' implementation? Or maybe I'm missing something?

[TimvdLippe]

There was a lot of discussion about this feature in the corresponding PR: #1619. That should hopefully clear up why we were forced to implement it this way 😢

[marius-bardan]

got it. so is there a way, that you know of, to invoke that for all tests, without me going into every test and adding that code? Maybe via some Test Listener or reflection?

[TimvdLippe]

You can write your own test runner that extends from the default Mockito runner and implements the logic.

We might actually consider that as part of the default mockito runner implementation. Do you mind opening a PR to start that discussion? It will mostly revolve around potential issues with multithreaded code.

[joshallenit]

we just updated our testing dependencies to latest version and ran into this OOM.
how is adding an '@after'-annotated function to 3000+ tests practical, of which some already have their own implementation '@after' implementation? Or maybe I'm missing something?

Classes can have as many @After functions as they want. In my case we had a base class that all classes inherited from.