Java heap usage disparity

[eclipsewebmaster]

| --- | --- |
| Bugzilla Link | 410113 |
| Status | NEW |
| Importance | P3 normal |
| Reported | Jun 06, 2013 13:35 EDT |
| Modified | Jun 27, 2013 05:24 EDT |
| Version | 1.2 |

[eclipsewebmaster]

By Jeff Hayes on Jun 06, 2013 13:56

The heap dump was produced under JZOS as a .phd running under heap size of -Xms2250m and -Xmx2250m. The MAT UI shows heap total as 1.7G. What would explain such a large disparity? I would expect the total to show as 2.25G or close to it.

see attached screenshot.

[eclipsewebmaster]

By Jeff Hayes on Jun 06, 2013 13:57

Created attachment 232057
screenshot

mat_bug.png

[eclipsewebmaster]

By Andrew Johnson on Jun 06, 2013 17:55

Does a system dump produce better results? If the problem is reproducible you could set up the JVM to generate system dumps for the same events as the PHD+Javacore.
You could then open both dumps, and use MAT comparison to compare the two.

PHD files have no information about the size of classes rather than objects, so that could make a difference.

[eclipsewebmaster]

By Andrew Johnson on Jun 27, 2013 05:24

I ran a simple test with -Xmx200m -Xdump:system+heap+java:events=systhrow+user,filter=java/lang/OutOfMemoryError,request=exclusive+prepwalk+compact

The PHD showed
Size: 162.7 MB Classes: 6.2k Objects: 1.3m Class Loader: 64
Class Name | Objects | Shallow Heap | Retained Heap
--------------------------------------------------------
java.lang.Class| 6,173 | 888,912 | \

The Core showed
Size: 168.7 MB Classes: 6.2k Objects: 1.2m Class Loader: 64
Class Name | Objects | Shallow Heap | Retained Heap
--------------------------------------------------------
java.lang.Class| 6,173 | 7,307,006 | \

so there is a bit of a difference in the size of the classes, as PHD files don't show the bytecode and jitted code sizes, whereas core files do.
The bytecode and jitted code might not be held on the heap though.

However the javacore showed:
1STGCHTYPE GC History
3STHSTTYPE 08:54:24:959653900 GMT j9mm.101 - J9AllocateIndexableObject() returning NULL! 78819504 bytes requested for object of class 0000000003300070 from memory space 'Flat' id=00000000029E16F8
3STHSTTYPE 08:54:24:959632881 GMT j9mm.134 - Allocation failure end: newspace=0/0 oldspace=39676672/209715200 loa=0/0
3STHSTTYPE 08:54:24:959632881 GMT j9mm.139 - Reference count end: weak=502 soft=12839 phantom=1 threshold=6 maxThreshold=32
3STHSTTYPE 08:54:24:959624941 GMT j9mm.91 - GlobalGC end: workstackoverflow=0 overflowcount=0 weakrefs=502 soft=12839 threshold=6 phantom=1 finalizers=396 newspace=0/0 oldspace=39676672/209715200 loa=0/0
3STHSTTYPE 08:54:24:959621205 GMT j9mm.90 - GlobalGC collect complete
3STHSTTYPE 08:54:24:959612330 GMT j9mm.94 - Class unloading end: classloadersunloaded=0 classesunloaded=0
3STHSTTYPE 08:54:24:959148528 GMT j9mm.60 - Class unloading start
3STHSTTYPE 08:54:24:959143390 GMT j9mm.137 - Compact end: bytesmoved=90346856
3STHSTTYPE 08:54:24:829834189 GMT j9mm.136 - Compact start: reason=compact to meet allocation
3STHSTTYPE 08:54:24:829833255 GMT j9mm.57 - Sweep end
3STHSTTYPE 08:54:24:829352171 GMT j9mm.56 - Sweep start
3STHSTTYPE 08:54:24:829351704 GMT j9mm.55 - Mark end
3STHSTTYPE 08:54:24:793408687 GMT j9mm.54 - Mark start
3STHSTTYPE 08:54:24:793403082 GMT j9mm.52 - GlobalGC start: weakrefs=502 soft=12839 phantom=1 finalizers=396 globalcount=266 scavengecount=0

so if the reason for your OutOfMemoryError was an failed allocation of 500MB then that might explain the failure.
Also, OutOfMemoryError can be thrown for excessive GC, where so many garbage collections are run that the VM throws an exception rather than attempting to make progress when all the time is being spent doing GC.
Examine your javacore file or verbose GC logs.
http://www-03.ibm.com/systems/z/os/zos/tools/java/faq/javafaq.html