Add DirectMemory Consumption

[eclipsewebmaster]

| --- | --- |
| Bugzilla Link | 473493 |
| Status | NEW |
| Importance | P3 enhancement |
| Reported | Jul 24, 2015 07:28 EDT |
| Modified | Feb 01, 2023 06:20 EDT |
| Version | 1.5 |
| See also | Gerrit change https://git.eclipse.org/r/63307 |
| Reporter | Benoit Lacelle |

Description

Hello,

DirectMemory is taking more and more importance in InMemory libraries. However, it is very difficult to tackle memory-leak issues and understand properly memory consumption in such applications as MAT analyses only the Heap.

Then, I would like to be able to associate DirectByteBuffer they direct-memory footprint in addition to their Heap.

I successfully done it by considering DirectByteBuffer instances as arrays and registering their capacity as heap footprint.

However, this breaks some unit-tests as the computed Heap size is not aligned anymore with the heap-size given pre-computed in the Heap file.

Does the MAT team consider such a feature useful?
What would be the proper way to implement it? In my implementation, which is just a POC for now, I changed the heap size associated to DirectByteBuffer instances.

Best

[eclipsewebmaster]

By Benoit Lacelle on Jul 28, 2015 11:58

The issue in the tests has been fixed. I prepared a branch with the modifications (very few of them as of now). But Gerrit rejects me.

How could I push this branch? This would at least enable me working on this through my different dev env.

Thanks

[eclipsewebmaster]

By Krum Tsvetkov on Sep 22, 2015 10:39

Hi Benoit,

Sorry for the delayed response. I find the proposal interesting, and would like to see what you did to achieve this.

Can you give any details about the error you get from Gerrit? I would try to help if I can.

Regards,
Krum

[eclipsewebmaster]

Dec 28, 2015 07:14

New Gerrit change created: https://git.eclipse.org/r/63307

[eclipsewebmaster]

By Kevin Grigorenko on Dec 28, 2015 10:45

Hi Benoit, You may be interested in the free IBM Extensions for Memory Analyzer (IEMA): http://www.ibm.com/developerworks/java/jdk/tools/iema/

IEMA includes a plugin under Open Query Browser > IBM Extensions > Java SE Runtime > DirectByteBuffers. This produces a report about DirectByteBuffers, including how many are only Phantomly Retained, the fact that Oracle SDKs less than V7 add a page size to the underlying malloc, etc.

Here is a sample report:

Alignment size of 4096 (only applies for Java < 7) bytes, and word size of 8 bytes.
6 instances of java.nio.DirectByteBuffer
6 non-viewed* DirectByteBuffers. Sum capacity (with overhead)=48 (48 B)
0 non-viewed*, phantomed*** DirectByteBuffers. Sum capacity (with overhead)=0 (0 B)
6 non-viewed*, non-phantomed** DirectByteBuffers. Sum capacity (with overhead)=48 (48 B)
Maximum non-viewed*, non-phantomed** DirectByteBuffer = 8 (8 B)

=> Sum DirectByteBuffers' (0) capacity only phantomly reachable: 0 (0 B)
=> Sum DirectByteBuffers' (6) capacity strongly, softly, or weakly reachable (i.e. not only phantomly reachable): 48 (48 B)

---------
Histogram of Incoming References to strongly referenced DBBs
6 instances incoming from java.nio.DirectLongBufferU=48 (48.0 B)

---

• A non-viewed DBB is one that has viewedBuffer=null. A viewed DBB is a duplicate or a splice of an existing DBB, which references the same underlying buffer.
  ** A non-phantomed DBB is one that has more than just a single sun.misc.Cleaner incoming reference.
  *** A phantomed DBB is one that has just a single sun.misc.Cleaner incoming reference.
  Example OQL: SELECT k, k.capacity FROM java.nio.DirectByteBuffer k WHERE ((viewedBuffer=null)and(inbounds(k).length>1))

"If the garbage collector determines at a certain point in time that the referent of a phantom reference is phantom reachable, then at that time or at some later time it will enqueue the reference. " (http://docs.oracle.com/javase/7/docs/api/java/lang/ref/PhantomReference.html).

[eclipsewebmaster]

By Benoit Lacelle on Feb 01, 2023 05:22

I missed this last comment. A few years later, I'm still interested in this. Where can one download given IEMA ? I see https://www.ibm.com/docs/en/support-assistant/5.0.0?topic=tools-extensions-memory-analyzer-tech-preview, but I can not find how to download such an extention for MAT.

[eclipsewebmaster]

By Andrew Johnson on Feb 01, 2023 06:20

See https://www.ibm.com/support/pages/eclipse-memory-analyzer-tool-dtfj-and-ibm-extensions\
This is a built with the extensions.

This shows holw to install them into an existing MAT:https://publib.boulder.ibm.com/httpserv/cookbook/Major_Tools-Eclipse_Memory_Analyzer_Tool.html#installation