JDK-8294677: chunklevel::MAX_CHUNK_WORD_SIZE too small for some applications

[tstuefe]

This patch is a workaround for a regression caused by the JEP-387 metaspace revamp. That revamp introduced a maximum limit to the size an individual metaspace allocation could have (4m).

4m was deemed much more than enough; normally metaspace allocations are tiny. But we now had a customer trying to load a generated giant class. The class was inefficiently generated but valid nonetheless.

The patch increases the maximum chunk size in Metaspace from 4M to 16M. This fixes the customer case mentioned in the JBS note. However, this is just a workaround since we still have a limit, albeit a larger one. Therefore I am working on a prototype that removes the limit completely (https://bugs.openjdk.org/browse/JDK-8300729) but that needs some more cooking time.

For details, please see JBS text as well as JDK-8300729.

---

Patch increases the maximum chunk size.

This affects the granularity used to reserve (not: commit - no RSS increase, just address space) metaspace memory, which in turn affects CompressedClassSpaceSize granularity (it gets silently rounded up to a multiple of root chunk size) as well as class space placement. Thankfully all of this is well abstracted behind Metaspace::reserve_alignment() so almost no changes were necessary. The increase from 4m to 16m is also small enough to be benign.

Increasing the maximum chunk size also increases the memory and runtime footprint of some tests that explicitly test metaspace chunk management. I checked all of these tests and changed some to keep the footprint low.

Finally, the patch uncovered some minor test errors in metaspace tests:

The gtest "metaspace:get_chunk_with_commit_limit" was supposed to test allocation from a metaspace chunk when hit by a commit limit. The limit was too high, effectively disabling the commit-failed path of this test. I expanded the test to test a variation of commit limits, not only one. The test also produced wrong positives with MetaspaceReclaimPolicy=none - those are fixed now (see comment). Note that I plan to remove the MetaspaceReclaimPolicy switch since, in practice, it seems of very limited use, and this would cut down on test variations and code complexity.

The gtest "metaspace:misc_sizes" tests that we can allocate the max. root chunk size (abstracted via Metaspace::max_allocation_word_size). A larger root chunk size caused us to hit the GC threshold now. I adjusted the test and expanded it to test both class space and non-class metaspace.

The elastic metaspace jtreg tests (test/hotspot/jtreg/runtime/Metaspace/elastic...) needed adaption since they have the root chunk size hard-coded. There is also a minor issue with using word size, but I left that untouched and opened https://bugs.openjdk.org/browse/JDK-8300732 to deal with this later.

---

Tests:

I manually ran hotspot_metaspace tests (a collection of metaspace, gtests, cds tests and some GC tests) on x64 and x86, fastdebug and release. GHAs are in process, and more tests are planned.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8294677: chunklevel::MAX_CHUNK_WORD_SIZE too small for some applications

Reviewers

• Volker Simonis (@simonis - Reviewer)
• Paul Hohensee (@phohensee - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/12115/head:pull/12115
$ git checkout pull/12115

Update a local copy of the PR:
$ git checkout pull/12115
$ git pull https://git.openjdk.org/jdk pull/12115/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 12115

View PR using the GUI difftool:
$ git pr show -t 12115

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/12115.diff

[bridgekeeper]

👋 Welcome back stuefe! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@tstuefe The following label will be automatically applied to this pull request:

• hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (7d5c5e2f)
• 00: Full (99b7b711)

[simonis]

Hi Thomas,

Looks good to me.

Thanks for fixing this quickly!

[openjdk]

@tstuefe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8294677: chunklevel::MAX_CHUNK_WORD_SIZE too small for some applications

Reviewed-by: simonis, phh

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 39 new commits pushed to the master branch:

• 1a3cb8c: 8296343: CPVE thrown on missing content-length in OCSP response
• 86fed79: 8300693: Lower the compile threshold and reduce the iterations of warmup loop in VarHandles tests
• 4525aa3: 8300867: Fix document issues in java.io
• a7f035d: 8300868: Reduce visibility in java.io.SerialCallbackContext
• 079255e: 8300864: Declare some fields in java.io as final
• a56598f: 8299684: (bf) JNI direct buffer functions with large capacity behave unexpectedly
• 542bfe6: 8300587: (bf) Some covariant overrides are missing @SInCE tags
• 03a9a88: 8300265: Remove metaprogramming/isSigned.hpp
• 5a4945c: 8299975: Limit underflow protection CMoveINode in PhaseIdealLoop::do_unroll must also protect type from underflow
• f307e8c: 8299795: Relativize locals in interpreter frames
• ... and 29 more: https://git.openjdk.org/jdk/compare/77f2d20e96712de725abffd9db5f28b1a48153b4...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[phohensee]

Lgtm.

[tstuefe]

Test errors seem unrelated. SAP nightlies are green (three days). I ran a number of tests manually to check that the memory footprint envelope does not change.

[tstuefe]

/integrate

[openjdk]

Going to push as commit 2292ce1.
Since your change was applied there have been 53 commits pushed to the master branch:

• 6dd8723: 8290918: Initial nroff manpage generation for JDK 21
• b3822f5: 8300589: Use @snippet and @linkplain in java.text.CollationKey and java.text.CompactNumberFormat
• afd5921: 8298610: Refactor archiving of ConstantPool::resolved_references()
• 937ba1c: 8300111: Add rpath for common lib locations for jpackageapplauncher
• 2da2e5a: 8300946: Add sun/security/provider/certpath/OCSP/OCSPNoContentLength to ProblemList
• 0323609: 8300706: Use @snippet in java.text
• b5ee3d1: 8299497: Usage of constructors of primitive wrapper classes should be avoided in java.desktop API docs
• 77a5010: 8286775: Remove identical per-compiler definitions of unsigned integral jtypes
• f79e587: 8300828: Avoid unnecessary array fill after creation in com.sun.media.sound
• 56dc3b0: Merge
• ... and 43 more: https://git.openjdk.org/jdk/compare/77f2d20e96712de725abffd9db5f28b1a48153b4...master

Your commit was automatically rebased without conflicts.

[openjdk]

@tstuefe Pushed as commit 2292ce1.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.