New DDR Command: !findinstances

[thallium]

This command is needed to profile the JVM and optimize JVM features. The command will output all the objects created using the given class. A summary that includes the total number of live and dead objects for the given class will also be outputted.

Usage:

!findinstances <class_name>

[thallium]

@babsingh FYI

[thallium]

Design:

Use GCHeapRegionIterator and GCObjectHeapIterator to iterate all the heap objects. For a live object, we can check the
if the class name matches the given class. For a dead object, if we are able to get the class name, we can check it with the given class, otherwise we can also keep track of a list/count of such bad objects.

@keithc-ca does this look good to you?

[keithc-ca]

What information would this provide that isn't already available via the existing !objectsizeinfo command?

[dmitripivkine]

Please note you can not differentiate live and dead (dark matter) objects in the core by just walking them in general case.

[babsingh]

What information would this provide that isn't already available via the existing !objectsizeinfo command?

The output will include all the addresses to the objects for a specific class. The intent is to inspect the objects and derive statistics to understand the impact on JVM features. This data will be used to optimize JVM features. Example investigations: Which data structure to use for managing objects internally? If there are many duplicates, will there be benefits of interning such objects?

++@tajila, for other insights.

[keithc-ca]

If we decide there's value in pursuing this, I suggest !findinstances <class_name>. The basic help should be sufficient and a summary is available via !objectsizeinfo.

[babsingh]

Please note you can not differentiate live and dead (dark matter) objects in the core by just walking them in general case.

Discussed with @dmitripivkine.-Xdump:heap:request=exclusive+compact+prepwalk, compact should remove all unreachable objects from the heap before the dump file is generated. For core files created without compact, the object inspection jdmpview cmds can consider throwing a warning to run with compact for more accurate results by removing dead objects.

[tajila]

The only other thing I would add is a option to include subclasses as well. Ie. If I want all method handles BoundedMethodHandle, ConstantHandle, FoldHandle, ... it would be nice to have a single query for that.

[dmitripivkine]

For a dead object, if we are able to get the class name, we can check it with the given class, otherwise we can also keep track of a list/count of such bad objects.

I think there is another confusion. As discussed above alive object is undistinguishable from dead object (dark matter). Iterator returns object pointer (as "alive"). Dead object in Iterator's terminology is not an object at all. This is "memory hole" (may be in a form of single slot hole or multi slot hole aka Linked Free Header). There is no class pointer or another object attribute. So, you can't do this.
If you need more explanations please do not hesitate to ask.

[thallium]

@dmitripivkine Here's my understanding (correct me if I'm wrong): there are three types of "objects": alive objects, dark matters, and holes. And an iterator can only return either an alive object or a dark matter.

My questions: what is a dark matter? If we can't distinguish alive object and dark matter, what does ObjectModel.isHoleObject(object) and ObjectModel.isDarkMatterObject(object) do? Thanks!

[dmitripivkine]

@dmitripivkine Here's my understanding (correct me if I'm wrong): there are three types of "objects": alive objects, dark matters, and holes. And an iterator can only return either an alive object or a dark matter.

My questions: what is a dark matter? If we can't distinguish alive object and dark matter, what does ObjectModel.isHoleObject(object) and ObjectModel.isDarkMatterObject(object) do? Thanks!

Dark matter is a dead object left GC in the heap due it's space can't by coalesced with other free spaces and it's size is not large enough to be added to the memory pool, so it is left as-is. Dark matter object should have its RAM class intact to allow GC to detect its size. The references from dark matter object can be incorrect (stale).
An Iterator can return object (alive or dark matter - you can not find a difference in the core) or pointer to free memory (organized as a single or multi slot holes). The functions you mentioned have deal with "holes" (ex. do correctness verification).

[thallium]

a summary is available via !objectsizeinfo.

@keithc-ca so you mean that we don't need to output the summary info right?

[keithc-ca]

so you mean that we don't need to output the summary info right?

Yes, unless !findinstances -s would do something substantially different than !objectsizeinfo.

[thallium]

For core files created without compact

@keithc-ca Is this possible to know?

[keithc-ca]

Sure, technically it is possible to determine if a specific object is reachable (see !isobjectalive), but it may not be practical to perform that test for every instance that !findinstances might consider reporting.

[thallium]

Sorry I didn't make myself clear, I mean is it possible to know if the compact request is presented or not?

[keithc-ca]

The J9RAS_DUMP_DO_COMPACT_HEAP bit will be set in the requestMask field of the agent that created the system dump if compact was specified, but finding the pointer to that agent may be challenging. See the !showdumpagents command that prints those request masks.

We could store the active agent in a new field of some globally accessible structure (e.g. RasDumpGlobalStorage), but that won't help with existing system dumps.

[thallium]

Then I guess it doesn't really worth pursuing this feature? We can just print a note on this. @babsingh thoughts?

[keithc-ca]

It may still be worth proceeding. The suggest warning could be omitted if we can't tell whether compact was specified, but making a link to the agent readily available could help future system dumps (and possibly other features).

The people doing the profiling that this is intended to support should know to use the compact request option.

[babsingh]

MemberName (MN) Usage

Benchmark	Total MNs	Unique MNs	Duplication
Micro Benchmark (1 Million MHs)	186272	34	99.9%
HeapHogLoadTest	27918	992	96%
Libert Startup JDK11	3588	2287	36%
Liberty Throughput JDK11	1568	1042	34%
Liberty Throughput JDK21	3653	2317	37%
Nashorn MathTest	8182	4799	41%
Nashorn LoadJS	5838	3226	45%

MemberName.vmtarget points to the resolved J9Method, and it is used to evaluate if MemberNames are the same. Duplication refers to the percentage of MemberNames which are repeated.

The above data is to support the HashTable approach to intern MemberNames. This will replace the linked list per class approach to amend MemberNames during HCR. @ThanHenderson Can you open an issue to document the design details and benefits of the HashTable approach?

fyi,
VM: @ThanHenderson @tajila
GC: @dmitripivkine @amicic
JIT: @nbhuiyan @jdmpapin @0xdaryl

[keithc-ca]

Please update the description to reflect the new proposed command name.

[babsingh]

Closing, completed with #19651.