JDK-8260282: Add option to compress heap dumps created by -XX:+HeapDumpOnOutOfMemoryError #2222

schmelter-sap · 2021-01-25T16:07:24Z

This change adds the optiont to created a gzipped heap dump by -XX:+HeapDumpOnOutOfMemoryError.

-XX:HeapDumpGzipLevel= sets the compression level. 0 (the default) means no gzipping of the dump. Otherwise the level has to be between 1 and 10.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8260282: Add option to compress heap dumps created by -XX:+HeapDumpOnOutOfMemoryError

Reviewers

Chris Plummer (@plummercj - Reviewer)
Thomas Stuefe (@tstuefe - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/2222/head:pull/2222
$ git checkout pull/2222

…fMemoryError

bridgekeeper · 2021-01-25T16:08:32Z

👋 Welcome back rschmelter! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-01-25T16:09:57Z

@schmelter-sap The following label will be automatically applied to this pull request:

hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2021-01-25T16:16:55Z

Webrevs

plummercj · 2021-01-25T19:23:05Z

src/hotspot/share/runtime/globals.hpp

+          "When HeapDumpOnOutOfMemoryError is on, the gzip compression "    \
+          "level of the dump file. 0 (the default) disables gzip "          \
+          "compression. Otherwise the level must be between 1 and 10.")     \
+          range(0, 10)                                                      \


Isn't the range 0-9? Although HeapDumper::dump() imposes no upper limit (it seems it should), the HeapDumpDCMD sets the upper limit to 9. See HeapDumpDCmd::execute().

You're right. 9 is the maximum compression level supported by zlib. I will update the PR.

plummercj

You need to run the test with a limited sized heap. Otherwise it may impact other tests, and would need to be moved into the resourcehogs directory.

schmelter-sap · 2021-01-26T07:57:42Z

You need to run the test with a limited sized heap. Otherwise it may impact other tests, and would need to be moved into the resourcehogs directory.

I've added a -Xmx128M. This should probably also done to TestHeapDumpOnOutOfMemoryError, which uses the same code.

plummercj · 2021-01-26T08:08:47Z

I've added a -Xmx128M. This should probably also done to TestHeapDumpOnOutOfMemoryError, which uses the same code.

Yes, it should. Probably it hasn't caused failures because my recollection is that it usually takes 2 or more such heap exhaustion tests running in parallel to result in unexpected failures in our testing, but it really depends on the test host and available memory. Also, the metaspace version of the test does use -XX:MaxMetaspaceSize=16m, so it doesn't use that much memory.

openjdk · 2021-01-26T08:10:07Z

@schmelter-sap This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8260282: Add option to compress heap dumps created by -XX:+HeapDumpOnOutOfMemoryError

Reviewed-by: cjplummer, stuefe

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 44 new commits pushed to the master branch:

11a70d1: 8260426: awt debug_mem.c DMem_AllocateBlock might leak memory
396a496: 8260467: Move well-known classes from systemDictionary.hpp to vmClasses.hpp
62ffe75: Merge
e28e111: 8260370: C2: LoopLimit node is not eliminated
7030d2e: 8260200: G1: Remove unnecessary update in FreeRegionList::remove_starting_at
c7661ae: 8260561: [doc] HexFormat has incorrect @SInCE tag
c52212b: Merge
408772c: 8259025: Record compact constructor using Objects.requireNonNull
81e730e: 8259276: C2: Empty expression stack when reexecuting tableswitch/lookupswitch instructions after deoptimization
c5ab7c3: 8260284: C2: assert(_base == Int) failed: Not an Int
... and 34 more: https://git.openjdk.java.net/jdk/compare/c538cd8b79c80d437100216dfae4e4f0f88798f0...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

plummercj · 2021-01-26T19:25:57Z

I've added a -Xmx128M. This should probably also done to TestHeapDumpOnOutOfMemoryError, which uses the same code.

Yes, it should. Probably it hasn't caused failures because my recollection is that it usually takes 2 or more such heap exhaustion tests running in parallel to result in unexpected failures in our testing, but it really depends on the test host and available memory. Also, the metaspace version of the test does use -XX:MaxMetaspaceSize=16m, so it doesn't use that much memory.

Looks like this was just addressed today by JDK-8252545.

tstuefe

Looks good. Nitpickings below, but I leave it up to you which ones you follow up to. Otherwise fine.

..Thomas

tstuefe · 2021-01-28T08:31:41Z

test/hotspot/jtreg/runtime/ErrorHandling/TestGZippedHeapDumpOnOutOfMemoryError.java

+ * @test
+ * @summary Test verifies that -XX:HeapDumpGzipLevel=0 works
+ * @library /test/lib
+ * @run driver/timeout=240 TestGZippedHeapDumpOnOutOfMemoryError run 0


Is the timeout needed? Should the oome not be pretty much instantaneous?

The timeout is not for the VM which creates the OOM, but the VM which starts that VM and then parses the created hprof file. I took the timeout from the TestHeapDumpOnOutOfMemoryError test, ,which seemed to run fine with this value.

tstuefe · 2021-01-28T08:31:41Z

test/hotspot/jtreg/runtime/ErrorHandling/TestGZippedHeapDumpOnOutOfMemoryError.java

+ * @summary Test verifies that -XX:HeapDumpGzipLevel=1 works
+ * @library /test/lib
+ * @run driver/timeout=240 TestGZippedHeapDumpOnOutOfMemoryError run 1
+ */


If the tests are fast enough, I would add at least another one at the end of the valid gzip range.

I don't think that this is needed, since these test should not test the gzipping (there are already tests for that).

tstuefe · 2021-01-28T08:31:41Z

test/hotspot/jtreg/runtime/ErrorHandling/TestGZippedHeapDumpOnOutOfMemoryError.java

+        String heapdumpFilename = "java_pid" + proc.pid() + ".hprof" + (level > 0 ? ".gz" : "");
+        OutputAnalyzer output = new OutputAnalyzer(proc);
+        output.stdoutShouldNotBeEmpty();
+        output.shouldContain("Dumping heap to " + heapdumpFilename);


Could this be a full path? In case printing on the VM side is ever changed to print the full path, I'd make this more lenient. Maybe two separate contains expressions or a shouldMatch with ".*" int he middle.

I think that if the VM output changes, it is OK to adjust the test. Making it more lenient could also make it less accurate.

schmelter-sap · 2021-01-29T13:53:08Z

/integrate

openjdk · 2021-01-29T13:54:04Z

@schmelter-sap Since your change was applied there have been 68 commits pushed to the master branch:

a5fb517: 8259404: Shenandoah: Fix time tracking in parallel_cleaning
0675473: 8252409: JShell: Intersection types cause NoSuchFieldError
64a150c: 6986863: ProfileDeferralMgr throwing ConcurrentModificationException
ea2c447: 8259778: Merge MutableSpace and ImmutableSpace
251c641: 8259487: Remove unused StarTask
1519632: 8251462: Simplify compilation policy
71128cf: 8260586: Shenandoah: simplify "Concurrent Weak References" logging
81e9e6a: 8260461: Modernize jsr166 tck tests
2b166d8: 8260596: Comment cleanup in BigInteger
1de3c55: 8260584: Shenandoah: simplify "Concurrent Thread Roots" logging
... and 58 more: https://git.openjdk.java.net/jdk/compare/c538cd8b79c80d437100216dfae4e4f0f88798f0...master

Your commit was automatically rebased without conflicts.

Pushed as commit d2b0ea1.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

schmelter-sap added 3 commits Jan 25, 2021

Added option to compress the heap dump created by -XX:+HeapDumpOnOutO…

21e64e9

…fMemoryError

Change allowed levels. 0 now means disabled.#

c644dff

Merge branch 'master' of https://github.com/openjdk/jdk into JDK-8260282

512c9af

openjdk bot added the hotspot-runtime label Jan 25, 2021

Fixed trailing whitespace.

88b00e0

openjdk bot added the rfr label Jan 25, 2021

plummercj reviewed Jan 25, 2021

View changes

schmelter-sap added 2 commits Jan 25, 2021

Fixed maximum supported compression level.

86478e2

Merge branch 'master' of https://github.com/openjdk/jdk into JDK-8260282

3bf0268

plummercj reviewed Jan 25, 2021

View changes

schmelter-sap added 2 commits Jan 26, 2021

Limit memory of heap dumping VM

92833c6

Merge branch 'master' of https://github.com/openjdk/jdk into JDK-8260282

471967f

plummercj approved these changes Jan 26, 2021

View changes

openjdk bot added the ready label Jan 26, 2021

tstuefe approved these changes Jan 28, 2021

View changes

openjdk bot closed this Jan 29, 2021

openjdk bot added integrated and removed ready rfr labels Jan 29, 2021

JDK-8260282: Add option to compress heap dumps created by -XX:+HeapDumpOnOutOfMemoryError #2222

JDK-8260282: Add option to compress heap dumps created by -XX:+HeapDumpOnOutOfMemoryError #2222

schmelter-sap commented Jan 25, 2021 •

edited by openjdk[bot] bot

bridgekeeper bot commented Jan 25, 2021

openjdk bot commented Jan 25, 2021

mlbridge bot commented Jan 25, 2021 •

edited

plummercj Jan 25, 2021

schmelter-sap Jan 25, 2021

plummercj left a comment

schmelter-sap commented Jan 26, 2021

plummercj commented Jan 26, 2021

openjdk bot commented Jan 26, 2021 •

edited

plummercj commented Jan 26, 2021

tstuefe left a comment

tstuefe Jan 28, 2021

schmelter-sap Jan 28, 2021

tstuefe Jan 28, 2021

schmelter-sap Jan 28, 2021

tstuefe Jan 28, 2021

schmelter-sap Jan 28, 2021

schmelter-sap commented Jan 29, 2021

openjdk bot commented Jan 29, 2021

JDK-8260282: Add option to compress heap dumps created by -XX:+HeapDumpOnOutOfMemoryError #2222

JDK-8260282: Add option to compress heap dumps created by -XX:+HeapDumpOnOutOfMemoryError #2222

Conversation

schmelter-sap commented Jan 25, 2021 • edited by openjdk[bot] bot

Progress

Issue

Reviewers

Download

bridgekeeper bot commented Jan 25, 2021

openjdk bot commented Jan 25, 2021

mlbridge bot commented Jan 25, 2021 • edited

Webrevs

Choose a reason for hiding this comment

Choose a reason for hiding this comment

plummercj left a comment

schmelter-sap commented Jan 26, 2021

plummercj commented Jan 26, 2021

openjdk bot commented Jan 26, 2021 • edited

plummercj commented Jan 26, 2021

tstuefe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

schmelter-sap commented Jan 29, 2021

openjdk bot commented Jan 29, 2021

schmelter-sap commented Jan 25, 2021 •

edited by openjdk[bot] bot

mlbridge bot commented Jan 25, 2021 •

edited

openjdk bot commented Jan 26, 2021 •

edited