Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OutOfMemoryError in ReallySimpleSwingDemoTest #227

Open
johanhaleby opened this issue Jul 23, 2015 · 45 comments
Open

OutOfMemoryError in ReallySimpleSwingDemoTest #227

johanhaleby opened this issue Jul 23, 2015 · 45 comments
Labels

Comments

@johanhaleby
Copy link
Collaborator

From hugo.josefson.old@gmail.com on November 24, 2009 19:44:46

What steps will reproduce the problem? 1. checkout entire subversion repo
2. cd trunk
3. mvn install

What is the expected output?
BUILD SUCCESSFUL

What do you see instead?
Running samples.junit4.swing.ReallySimpleSwingDemoTest
org.apache.maven.surefire.booter.SurefireExecutionException: Java heap
space; nested exception is java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
at javassist.bytecode.CodeAttribute.(CodeAttribute.java:105)
at javassist.bytecode.AttributeInfo.read(AttributeInfo.java:78)
at javassist.bytecode.MethodInfo.read(MethodInfo.java:498)
at javassist.bytecode.MethodInfo.(MethodInfo.java:79)
at javassist.bytecode.ClassFile.read(ClassFile.java:716)
at javassist.bytecode.ClassFile.(ClassFile.java:85)
at javassist.CtClassType.getClassFile2(CtClassType.java:190)
at javassist.CtClassType.makeFieldCache(CtClassType.java:760)
at javassist.CtClassType.getMembers(CtClassType.java:751)
at javassist.CtClassType.getMethod0(CtClassType.java:1049)
at javassist.CtClassType.getMethod(CtClassType.java:1038)
at javassist.expr.MethodCall.getMethod(MethodCall.java:114)
at
org.powermock.core.transformers.impl.MainMockTransformer$PowerMockExpressionEditor.edit(MainMockTransformer.java:237)
at javassist.expr.ExprEditor.loopBody(ExprEditor.java:191)
at javassist.expr.ExprEditor.doit(ExprEditor.java:90)
at javassist.CtClassType.instrument(CtClassType.java:1289)
at
org.powermock.core.transformers.impl.MainMockTransformer.transform(MainMockTransformer.java:70)
at
org.powermock.core.classloader.MockClassLoader.loadMockClass(MockClassLoader.java:204)
at
org.powermock.core.classloader.MockClassLoader.loadModifiedClass(MockClassLoader.java:146)
at
org.powermock.core.classloader.DeferSupportingClassLoader.loadClass(DeferSupportingClassLoader.java:61)
at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:399)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:700)
at java.lang.ClassLoader.defineClass(ClassLoader.java:545)
at
org.powermock.core.classloader.MockClassLoader.loadMockClass(MockClassLoader.java:211)
at
org.powermock.core.classloader.MockClassLoader.loadModifiedClass(MockClassLoader.java:146)
at
org.powermock.core.classloader.DeferSupportingClassLoader.loadClass(DeferSupportingClassLoader.java:61)
at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:399)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

Please refer to
/Users/hugo/code/other/powermock/trunk/modules/module-test/powermock/junit4-test/target/surefire-reports
for the individual test results.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 minutes 19 seconds
[INFO] Finished at: Tue Nov 24 19:37:18 CET 2009
[INFO] Final Memory: 42M/94M
[INFO] ------------------------------------------------------------------------

Additional information:

[550:trunk]$ echo $MAVEN_OPTS
-Xmx3072m -XX:MaxPermSize=768m

Original issue: http://code.google.com/p/powermock/issues/detail?id=207

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on November 30, 2009 07:25:37

This is a known problem and we're going to work with it on Thursday!

Status: Accepted
Labels: -Priority-Medium Priority-Critical Milestone-Release1.4

@johanhaleby
Copy link
Collaborator Author

From rlag...@gmail.com on January 03, 2010 06:37:23

Hi!

i have the same problem when running my tests. Is it possible to get any date
estimation how long the fix will take?

Thanks

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on January 09, 2010 07:32:12

Unfortunately I don't. We still haven't managed to find out what's causing the memory
leak. However our goal is to have it fixed when we release version 1.4.

@johanhaleby
Copy link
Collaborator Author

From kristof....@gmail.com on May 17, 2010 04:41:36

We are also getting java.lang.OutOfMemory problems when running powermock tests with
maven. It seams to me there's some problem with MockClassLoader since a memory dump
tells me a total of 19 MockClassLoader instances exist. I wonder whether those
instances shouldn't be cleaned up?

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on May 17, 2010 04:53:01

They should indeed! We haven't managed to find out why the MockCL isn't GC:ed though.
If you have any ideas please let us know.

@johanhaleby
Copy link
Collaborator Author

From kristof....@gmail.com on May 17, 2010 06:21:19

The problem I was having, was that spring and log4j classes were also loaded by the
MockClassLoader, causing spring and log4j classes to be loaded multiple times by
multiple instances of MockClassLoader.
The solution for our project was to use the PowerMockIgnore annotation signaling that
spring and log4j classes should be loaded by the system ClassLoader by default:
@PowerMockIgnore( { "org.springframework.*", "org.apache.log4j" })

I wonder why all classes (except the deferred ones) are actually loaded by the
MockClassLoader? Wouldn't it be sufficient to only load mock classes using the
MockClassLoader and delegate classloading of all other classes to the system ClassLoader?

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on May 17, 2010 06:34:34

Good question. To be honest I don't really remember why. We must refactor the
MockClassloader because it's a huge mess right now.

@johanhaleby
Copy link
Collaborator Author

From Alexandr...@gmail.com on June 24, 2010 01:56:39

any progress on this ? :(

@johanhaleby
Copy link
Collaborator Author

From kristof....@gmail.com on June 24, 2010 04:27:37

I think I understand by now why all other classes are loaded by default by the MockClassLoader. I think it's because classes use the ClassLoader by which they are loaded themselves to find other classes they use. That way, you make sure the mocked classes are loaded by the MockClassLoader. By using the PowerMockIgnore annotation, you can tell Powermock: Hey, I'm sure this class is not using my mocked class underneath, so you can load it by the system ClassLoader.

At least, that's my interpretation. I'm not 100% sure I'm right on this.
However, the problem remains: MockClassLoaders aren't garbage collected and keep references to a lot of other objects (in ClassPool). Maybe just cleaning up those references when tearing down the test could help?

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on July 14, 2010 08:05:13

I've experimented a bit with clearing the ClassPool in the past but I couldn't find a good solution. I think I ended up with ClassNotFoundException's in some subsequent tests when running in a suite.

@johanhaleby
Copy link
Collaborator Author

From deepsh...@gmail.com on July 17, 2010 00:27:37

For me the @PowerMockIgnore fixed the issue for a few days. Later the OutOfMemoryError re-appeared. Surprisingly, this issue only appear for me on a 64 bit JDK, it did not appear on 32 bit JDK. Installed the 32 bit JDK, ran all the tests and bingo! All of them pass. Visit this link for details. http://www.gitshah.com/2010/07/how-to-fix-outofmemoryerror-when_17.html

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on July 22, 2010 02:25:06

Labels: -Milestone-Release1.4 Milestone-Release1.5

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on July 22, 2010 05:14:17

While I haven't solved the problem there has at least been some progress today. The OutOfMemoryError no longer occurs for me after I've clean up some state in PowerMockRunner (by setting suiteChunker to null in AbstractCommonPowerMockRunner#run method). I'll include this fix in 1.4 and you can try it out if it makes any difference for you.

@johanhaleby
Copy link
Collaborator Author

From jussi.ni...@gmail.com on September 08, 2010 05:51:46

Just made some profiling for this and it seems that mockClassLoader is created for each test and is kept in PermGen. This was with version 1.4.5 (powermock-module-junit4 and powermock-api-mockito). Hope you get this fixed soon!

@johanhaleby
Copy link
Collaborator Author

From marci...@gmail.com on October 25, 2010 13:49:31

This issue happened with me when mocking static final methods using the @PrepareForTest annotation on the METHOD.
When i moved the @PrepareForTest to the CLASS, then the error stopped.

Maybe works as a workaround for you guys.
hope this gets fixed soon!

@johanhaleby
Copy link
Collaborator Author

From ilinca.v...@gmail.com on November 18, 2010 05:13:08

I'm using powermock 1.4.5, and I still have problems. Moved my @PrepareForTest statement to class instead of method, and while I don't get the same OutOfMemory error, some methods seem to not pick up on the PrepareForTest annotation, meaning I have to declare the same annotation, I have for the class for certain methods anyway, if I want to avoid getting assertion errors.

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on December 27, 2010 01:24:17

We've instigated this a bit further and we think that the problem lies not in PowerMock but in EasyMock (and possibly Mockito but we haven't looked at Mockito yet).

I've created a patch for EasyMock to solve one of the two problems that we've found. The issue is here: https://sourceforge.net/tracker/?func=detail&aid=3145906&group_id=82958&atid=567837

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on December 27, 2010 04:58:23

Just committed code to PowerMock that cleans out some of EasyMocks state after each test method. This seems to reduce memory consumption by 75% (without applying the EasyMock patch mentioned above).

@johanhaleby
Copy link
Collaborator Author

From Alexandr...@gmail.com on December 28, 2010 00:16:39

i have encountered this problem with mockito, so it's there too.

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on December 28, 2010 00:35:51

Yes there's a problem with Mockito as well. I tried to do a similar thing in Mockito as with EasyMock but it doesn't seem to work all that well. I've managed to clean up almost everything but there's one class that prevents the classloader from being GC:ed and that's a PowerMock class called NewInvocationSubstitute. To my understanding the reason for it is that CGLib holds a reference to the classloader. But no matter how much I try I haven't found a way to solve it without failing the tests.

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on December 28, 2010 00:40:13

Still it's better now than it was before. The only time you'll run into trouble now (I think) is when you mock new using (whenNew) because that's when the NewInvocationSubstitute is used.

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on April 10, 2011 05:51:49

Labels: -Milestone-Release1.5 Milestone-Release1.6

@johanhaleby
Copy link
Collaborator Author

From dmeibu...@gmail.com on September 09, 2011 19:02:22

See also related Issue 346 .

@johanhaleby
Copy link
Collaborator Author

From mike.kol...@gmail.com on September 14, 2011 22:48:23

When will this be fixed for Mockito?

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on September 14, 2011 23:14:09

I fixed several OOM issues related to PowerMock and Mockito a while ago. It takes a tremendous amount of time (at least for me) to track them down and fix them. I don't remember everything so clearly now but I think there was one issue OOM that I failed to resolve last time I checked that had to do with new instance mocking. Is this the issue you're experiencing?

@johanhaleby
Copy link
Collaborator Author

From mike.kol...@gmail.com on September 14, 2011 23:49:08

We are using PowerMock + Mockito to mock out calls to static methods in our tests.

We were getting lots of these errors in mvn build output, which we first ignored:

log4j:ERROR A "org.apache.log4j.xml.DOMConfigurator" object is not assignable to a "org.apache.log4j.spi.Configurator" variable.
log4j:ERROR The class "org.apache.log4j.spi.Configurator" was loaded by
log4j:ERROR [org.powermock.core.classloader.MockClassLoader@49e808ca] whereas object of type
log4j:ERROR "org.apache.log4j.xml.DOMConfigurator" was loaded by [sun.misc.Launcher$AppClassLoader@5acac268].
log4j:ERROR Could not instantiate configurator [org.apache.log4j.xml.DOMConfigurator].
log4j:WARN No appenders could be found for logger (org.hibernate.validator.util.Version).
log4j:WARN Please initialize the log4j system properly.

After a few more tests were added, the build was starting to fail because of:
org.apache.maven.surefire.booter.SurefireExecutionException: PermGen space; nested exception is java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.testAborted(PowerMockJUnit44RunnerDelegateImpl.java:232)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:210)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:161)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:135)
at org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:34)
at org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:133)
at org.powermock.modules.junit4.common.internal.impl.JUnit4TestSuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:112)
at org.powermock.modules.junit4.common.internal.impl.AbstractCommonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:55)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)

We tried using @PowerMockIgnore("org.apache.log4j.*"), but that didn't help much.
Basically only workaround that works for us now is to run this module's tests with fork=true so that each runs in a new jvm, but this isn't such a good solution.

@johanhaleby
Copy link
Collaborator Author

From mason....@gmail.com on November 09, 2011 23:34:02

I also getting the following error, when using powermock-mockito-junit-1.4.10 suit。

log4j:ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by
log4j:ERROR [org.powermock.core.classloader.MockClassLoader@1a125f0] whereas object of type
log4j:ERROR "org.apache.log4j.ConsoleAppender" was loaded by [sun.misc.Launcher$AppClassLoader@19821f].
log4j:ERROR Could not instantiate appender named "console".

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on November 10, 2011 11:14:57

This has nothing to do with this issue. See FAQ #5.

@johanhaleby
Copy link
Collaborator Author

From vivek.na...@gmail.com on September 27, 2012 14:01:23

Has this issue been solved in the latest release of PowerMock?

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on September 27, 2012 22:24:07

If I remember it correctly there was one memory leak that I couldn't track down but I fixed most of them. I think that's why the issue has not yet been closed.

@johanhaleby
Copy link
Collaborator Author

From michal.r...@gmail.com on January 19, 2013 07:56:17

Are there any updates to this issue ?

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on January 20, 2013 11:08:53

I suppose you're running into OOM issues?

@johanhaleby
Copy link
Collaborator Author

From michal.r...@gmail.com on February 03, 2013 06:23:25

@johan: That's right - OOM. For 21 test classes with PowerMock runner, I have 21 different class definiotions...

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on February 03, 2013 08:56:24

Do you have any experience with tracking down OOM issues? In that case it'll be a huge help if you can try to find out what's happening. I remember I dug into some issues a couple of years ago and for me it took 2 days or so of my holiday to track things down and fix them (fixing was the easy part though). I don't see myself having the time to do something similar now in a foreseeable future :(

@johanhaleby
Copy link
Collaborator Author

From hen...@akarlsson.com on June 26, 2013 05:39:33

Johan: Please see https://code.google.com/p/mockito/issues/detail?id=437 for two issues with PermGen OOM from Powermock/Mockito. Maybe they can be solved from Powermock? Might not be the same issue as the original issue here, but at least they will fix some OOM problems. These two problems used about 62% of the memory when I analyzed a thread dump from our JUnit tests.

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on June 26, 2013 05:44:09

Thanks for the pointer! It would be really great if you (or anyone else) could help out tracking this down and perhaps even provide a patch.

@johanhaleby
Copy link
Collaborator Author

From hen...@akarlsson.com on June 27, 2013 03:32:29

I did some investigations regarding this and managed to solve the problem by:

  • Extending the Mockito class MethodInterceptorFilter and overriding the intercept method to clear the ThreadLocal for ThreadSafeMockingProgress and GlobalConfiguration if the intercepted method is "finalize" (functionality for clearing was already in MockitoStateCleaner in MockCreator).
  • Extracting the MockitoStateCleaner so that it could be used by both MockCreator and the new method interceptor filter.
  • Using the newly created method interceptor filter class in MockCreator instead of Mockito MethodInterceptorFilter.

This has solved most of our memory problems. Our test suite is now using almost half the memory it used to. There are however still leaks of javassist.ClassPool, I will see if I find some time to take a look at them as well.

Unfortunately I have some Maven problems with the project (I'm not used to Maven..) with failing tests in powermock-module-test-easymock-junit4 (where I have not changed anything), but I have attached the patch with the changes that I have made.

Attachment: patch.txt

@johanhaleby
Copy link
Collaborator Author

From hen...@akarlsson.com on June 27, 2013 08:28:08

It seems like the ClassPool leaks are from MockClassLoader "leaks", which in our test suite was due to two of our third party applications was leaving some ThreadLocals and other running threads. This prevented the MockClassLoader to be garbage collected.

@johanhaleby
Copy link
Collaborator Author

From johan.ha...@gmail.com on June 30, 2013 23:06:58

Thanks a lot for helping out Henrik! It's really invaluable! It would have taken me a great amount of time to track this down and fix so I'm really thankful. I've applied the patch to trunk now. Please try it out if you like.

@johanhaleby
Copy link
Collaborator Author

From hen...@akarlsson.com on August 06, 2013 01:35:09

I am now back from my vacation and have successfully tested the trunk version in our project. Thank you Johan for quickly adding the patch!

@johanhaleby
Copy link
Collaborator Author

From deepthi...@gmail.com on November 14, 2013 22:24:56

Hi,

I am using powermock version 1.4.10 with jmock/Easymock
Still i am able to see OutOfMemory error(GC overhead).
It would be helpful if u let me know whether i should still use @PowerMockIgnore annotation as a workaround or is it fixed in recent versions?

@johanhaleby
Copy link
Collaborator Author

From soul...@gmail.com on February 07, 2014 04:48:57

PowerMock 1.5.1 - still OutOfMemory.

But if I use 1.5.4 - everything is ok!

@johanhaleby
Copy link
Collaborator Author

From rasmusse...@gmail.com on June 23, 2014 05:46:32

What is the current status? Has this issue been fixed?

@gabrielbastien
Copy link

gabrielbastien commented Jun 15, 2017

I've just run into this same issue running:
powermock 1.7.0RC4
mockito 2.7.22
spring 3.1.2RELEASE
jre 1.7.0_09

I'm running a test suite that runs about 500 tests and my PermGen is filled with javaassist.ClassPool objects that take up ~4mb a piece. What is the fix for this issue? Should we be doing something specific in our test setup/tear down?

EDIT:
I just noticed the problem seems to go away (no PermGen memory error) when I use jre 1.8.0_102. But that could simply be because 1.8 doesn't use PermGen anymore. It doesn't mean the memory leak is gone.

@thekingn0thing
Copy link
Member

PowerMock uses a new class loader for each test class (or even test when @PrepareForTest is put on method level). Before Java 1.8 such behaviour leads to quick filling PermGen with default size. And only one way to avoid the issue – using flag -XX:PermSize=128m or more.

The issue became worse due to that all classes except system and that added to @PowerMockIgnore are loaded by MockClassLoader even if they are not modified. But PowerMock has to load all classes by the same class loader to avoid issue with package-private members.
You also can mitigate the issue with using @PowerMockIgnore and point as match package to ignore as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants