8327645: Serial heap dump should not consume double amount of disk space

[caoman]

Hi all,

Could anyone review this fix to make serial heap dump only write to a single file?
We highly appreciate the work in https://bugs.openjdk.org/browse/JDK-8306441. However, many of our customers still need to use serial heap dump, as they have limited disk space and need to dump to a network socket.

Tested:
Stress tested with a fastdebug build with tests in hotspot/jtreg/serviceability/dcmd/gc/HeapDump*.

-Man

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8327645: Serial heap dump should not consume double amount of disk space (Bug - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/18160/head:pull/18160
$ git checkout pull/18160

Update a local copy of the PR:
$ git checkout pull/18160
$ git pull https://git.openjdk.org/jdk.git pull/18160/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 18160

View PR using the GUI difftool:
$ git pr show -t 18160

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/18160.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back manc! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@caoman The following label will be automatically applied to this pull request:

• hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 02: Full (73172009)
• 01: Full - Incremental (a1b15ec9)
• 00: Full (64933e30)

[openjdk]

@caoman Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

[plummercj]

/label add serviceability

[openjdk]

@plummercj
The serviceability label was successfully added.

[jianglizhou]

The change seems reasonable and avoids some of the extra file overhead in the serial case.

[alexmenkov]

Heap dumper was switched to always use segments by JDK-8299426 / JDK-8321565.
I suppose your fix will cause broken heapdump if there are some unmounted virtual threads.
You can try to run test/hotspot/jtreg/serviceability/jvmti/vthread/HeapDump/VThreadInHeapDump.java

[caoman]

@alexmenkov Thank you for the quick feedback. I can reproduce the failure with VThreadInHeapDump.java and -XX:ActiveProcessorCount=1. The problem seems that the writer cannot write HPROF_FRAME and HPROF_TRACE records in the middle of dumping an HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT, so it resorts to a separate global writer.

Do we really need to keep all the HPROF_FRAME and HPROF_TRACE records located together? If not, two possible solutions are:

1. Keep track of all unmounted vthread oops, and dump their stack traces after finishing the HPROF_HEAP_DUMP_SEGMENT.
2. Write an HPROF_HEAP_DUMP_END, dump the vthread's stack trace, then start another HPROF_HEAP_DUMP_SEGMENT.

First approach seems resulting in a more organized heap dump. In any case, we probably only need to do this for serial heap dump. Parallel heap dump could keep using the current approach.

[alexmenkov]

Right, global writer is used to write HPROF_FRAME/HPROF_TRACE records when unmounted VT is found in the heap.
The spec does not require HPROF_FRAME/HPROF_TRACE to be in the beginning. Main concern here is compatibility as we don't control external hprof tools.
Solution #2 looks better to me.
It does not require extra memory to track unmounted VT (we may have millions) and it can prevent compatibility problem if we first write HPROF_FRAME/HPROF_TRACE records and then HPROF_GC_ROOT_THREAD_OBJ and stack reference subrecords which refer to the HPROF_TRACE

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[bridgekeeper]

@caoman This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[caoman]

Converted to draft until I have time to address the issue.

[bridgekeeper]

@caoman This pull request has been inactive for more than 8 weeks and will be automatically closed if another 8 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!