8267293: vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java fails when JTREG_JOBS > 25

[DamonFool]

Hi all,

vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java fails on our many-core machines due to -XX:MaxRAMPercentage=0.
This is because -XX:MaxRAMPercentage=0 will be 0 if JTREG_JOBS > 25 [1].

We can also reproduce the bug by: make test TEST="vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java" JTREG="JOBS=26" on almost all machines.

This fix will make it to be more robust, which is suggested by @shipilev [2] and many thanks to him.

Thanks.
Best regards,
Jie

[1] https://github.com/openjdk/jdk/blob/master/make/RunTests.gmk#L741
[2] #4062 (review)

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8267293: vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java fails when JTREG_JOBS > 25

Reviewers

• Thomas Stuefe (@tstuefe - Reviewer)
• Aleksey Shipilev (@shipilev - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4076/head:pull/4076
$ git checkout pull/4076

Update a local copy of the PR:
$ git checkout pull/4076
$ git pull https://git.openjdk.java.net/jdk pull/4076/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4076

View PR using the GUI difftool:
$ git pr show -t 4076

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4076.diff

[bridgekeeper]

👋 Welcome back jiefu! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@DamonFool The following label will be automatically applied to this pull request:

• hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (2e618c17)
• 00: Full (0838bac9)

[tstuefe]

Hi Jie,

You can safely get by with less. A very safe bet for all platforms would be:

• 256m heap
• 8m MaxMetaspaceSize

The latter could be probably reduced more (the smaller metaspace, the faster the test comes to conclusion), down to 6m or 4m. On my machine 4m works for both 64 and 32bit. But 8m is probably safe on all platforms.

Cheers, Thomas

[shipilev]

Yeah, this works on my machines:

diff --git a/test/hotspot/jtreg/vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java b/test/hotspot/jtreg/vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java
index 0d5f1a1626f..980c9303f04 100644
--- a/test/hotspot/jtreg/vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java
+++ b/test/hotspot/jtreg/vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java
@@ -36,7 +36,7 @@
  * @build vm.mlvm.anonloader.stress.oome.metaspace.Test
  * @run driver vm.mlvm.share.IndifiedClassesBuilder
  *
- * @run main/othervm -XX:-UseGCOverheadLimit -XX:MetaspaceSize=10m -XX:MaxMetaspaceSize=20m vm.mlvm.anonloader.stress.oome.metaspace.Test
+ * @run main/othervm -Xmx256m -XX:-UseGCOverheadLimit -XX:MaxMetaspaceSize=8m vm.mlvm.anonloader.stress.oome.metaspace.Test
  */
 
 package vm.mlvm.anonloader.stress.oome.metaspace;

[DamonFool]

Thanks @tstuefe and @shipilev .

The test time has been reduced from 41.8s to 34.8s .
And it passed on all our x86 platforms.
Updated.
Thanks.

[shipilev]

Seems like GHA is barfing up due to infrastructure issues. Please re-start the jobs, once current run finishes.

[DamonFool]

/test

[openjdk]

@DamonFool you need to get approval to run the tests in tier1 for commits up until 2e618c1

[shipilev]

"/test" command does not work. GHA start automatically on PR updates and/or with manual trigger: https://github.com/DamonFool/jdk/actions/workflows/submit.yml

[DamonFool]

"/test" command does not work. GHA start automatically on PR updates and/or with manual trigger: https://github.com/DamonFool/jdk/actions/workflows/submit.yml

Got it.
Thanks for your help. @shipilev

[DamonFool]

Seems like GHA is barfing up due to infrastructure issues. Please re-start the jobs, once current run finishes.

Hi @shipilev ,

Some jobs seem to be blocked.
But I think the unfinished jobs make no sense since they don't cover the affected changes at all.

Thanks.

[tstuefe]

LGTM

[openjdk]

@DamonFool This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8267293: vmTestbase/vm/mlvm/anonloader/stress/oome/metaspace/Test.java fails when JTREG_JOBS > 25

Reviewed-by: stuefe, shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 26 new commits pushed to the master branch:

• 70f6c67: 8233380: CHT: Node allocation and freeing
• 2563a6a: 8266962: Add arch supporting check for "Op_VectorLoadConst" before creating the node
• 4954383: 8267364: Remove mask.incr which is introduced by JDK-8256973
• c2b50f9: 8266480: Implicit null check optimization does not update control of hoisted memory operation
• 3f883e8: 8267351: runtime/cds/SharedBaseAddress.java fails on x86_32 due to Unrecognized VM option 'UseCompressedOops'
• 7aa6568: 8256973: Intrinsic creation for VectorMask query (lastTrue,firstTrue,trueCount) APIs
• 65a8bf5: 8265126: [REDO] unified handling for VectorMask object re-materialization during de-optimization
• ff84577: 8267098: AArch64: C1 StubFrames end confusingly
• 0daec49: 8267246: -XX:MaxRAMPercentage=0 is unreasonable for jtreg tests on many-core machines
• 324defe: 8267212: test/jdk/java/util/Collections/FindSubList.java intermittent crash with "no reachable node should have no use"
• ... and 16 more: https://git.openjdk.java.net/jdk/compare/cd1c17c0a6416a8d16cf2035f3e97dba95b6b8af...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[DamonFool]

Thanks @tstuefe and @shipilev .
/integrate

[openjdk]

@DamonFool Since your change was applied there have been 26 commits pushed to the master branch:

• 70f6c67: 8233380: CHT: Node allocation and freeing
• 2563a6a: 8266962: Add arch supporting check for "Op_VectorLoadConst" before creating the node
• 4954383: 8267364: Remove mask.incr which is introduced by JDK-8256973
• c2b50f9: 8266480: Implicit null check optimization does not update control of hoisted memory operation
• 3f883e8: 8267351: runtime/cds/SharedBaseAddress.java fails on x86_32 due to Unrecognized VM option 'UseCompressedOops'
• 7aa6568: 8256973: Intrinsic creation for VectorMask query (lastTrue,firstTrue,trueCount) APIs
• 65a8bf5: 8265126: [REDO] unified handling for VectorMask object re-materialization during de-optimization
• ff84577: 8267098: AArch64: C1 StubFrames end confusingly
• 0daec49: 8267246: -XX:MaxRAMPercentage=0 is unreasonable for jtreg tests on many-core machines
• 324defe: 8267212: test/jdk/java/util/Collections/FindSubList.java intermittent crash with "no reachable node should have no use"
• ... and 16 more: https://git.openjdk.java.net/jdk/compare/cd1c17c0a6416a8d16cf2035f3e97dba95b6b8af...master

Your commit was automatically rebased without conflicts.

Pushed as commit 2d407e1.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

[DamonFool]

Hi Jie,

You can safely get by with less. A very safe bet for all platforms would be:

• 256m heap
• 8m MaxMetaspaceSize

The latter could be probably reduced more (the smaller metaspace, the faster the test comes to conclusion), down to 6m or 4m. On my machine 4m works for both 64 and 32bit. But 8m is probably safe on all platforms.

Cheers, Thomas

Maybe -Xmx256m is not enough.
OOME on aarch64: https://bugs.openjdk.java.net/browse/JDK-8267404
Will do more investigation tomorrow.

[tstuefe]

Hmm... sorry for that. Weird though since I would not have expected a lot of platform dependency here (apart from 32 vs 64bit).

Ideally this test should be rewritten to load large classes, not these tiny ones, to get a better ratio between metaspace and heap space.

[DamonFool]

Hmm... sorry for that. Weird though since I would not have expected a lot of platform dependency here (apart from 32 vs 64bit).

Ideally this test should be rewritten to load large classes, not these tiny ones, to get a better ratio between metaspace and heap space.

Maybe, there is something special on Oracle's aarch64 platforms.
All our aarch64 machines fail to reproduce the OOME.

I've asked the help of @dcubed-ojdk to test with -Xmx1g and -Xmx2g.
If it still fails, I'd like to revert the change.

Thanks.