Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8276086: Increase size of metaspace mappings #6148

Conversation

tstuefe
Copy link
Member

@tstuefe tstuefe commented Oct 28, 2021

Disregarding class space, metaspace consists of a list of individual mappings. Traditionally, individual mappings had been rather small (2 root chunks => 8 MB). That is because the mechanism to release metaspace memory to the OS had been to unmap unused mappings, and the larger the mappings were the smaller the chance of them being completely unused, due to fragmentation.

We recently did away with this old release mechanism in favor of a new one with finer granularity (see https://bugs.openjdk.java.net/browse/JDK-8275582). So there is no motivation anymore to keep mappings particularly small.

There are however good reasons to enlarge them:

  • less fragmentation on the OS mapping level (e.g. on Linux, number of VMAs would go down, has a number of advantages)
  • Larger mappings -> fewer of them -> list gets smaller, which is good for functions iterating this list (e.g. Metaspace::contains(), see https://bugs.openjdk.java.net/browse/JDK-8275704).

The effect of larger mappings is an increase in the step size with process virtual size will grow. So, before vsize grew in 8MB steps, now those steps are larger and rarer. In total we should not use much more vsize than before. Larger mappings would have no effect on committed memory size.

And note that vsize is pretty much irrelevant with modern OSes. For our hotspot on 64-bit, it is astronomical anyway, so this change would have no impact in real life.

The only exception are 32-bit platforms, where address space itself is limited, so this change is limited to 64-bit.


How it looks after loading 3000 classes:

64-bit:

Virtual space:                                                                       
  Non-class space:      384,00 MB reserved,     338,94 MB ( 88%) committed,  6 nodes.
      Class space:        1,00 GB reserved,      13,00 MB (  1%) committed,  1 nodes.
             Both:        1,38 GB reserved,     351,94 MB ( 25%) committed.          

32-bit (note: no class space, smaller data):

Virtual space:                                                    
    248,00 MB reserved,     242,00 MB ( 98%) committed,  31 nodes.

Tests:

  • manual gtests + metaspace jtreg tests (32bit and 64bit)
  • GHAs
  • SAP nightlies

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/6148/head:pull/6148
$ git checkout pull/6148

Update a local copy of the PR:
$ git checkout pull/6148
$ git pull https://git.openjdk.java.net/jdk pull/6148/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 6148

View PR using the GUI difftool:
$ git pr show -t 6148

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/6148.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Oct 28, 2021

👋 Welcome back stuefe! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Oct 28, 2021

@tstuefe The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-runtime label Oct 28, 2021
@tstuefe tstuefe marked this pull request as ready for review Oct 28, 2021
@openjdk openjdk bot added the rfr label Oct 28, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 28, 2021

Webrevs

@GoeLin
Copy link
Member

@GoeLin GoeLin commented Oct 29, 2021

Hi Thomas, why did you choose 64MB for the new size?

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Oct 29, 2021

Hi Thomas, why did you choose 64MB for the new size?

No hard reason. I chose a number large enough to significantly reduce the list length, but small enough to not inflate vsize too much for very small apps. Most small java applications should get along fine with less and therefore would never need a second mapping.

@GoeLin
Copy link
Member

@GoeLin GoeLin commented Oct 29, 2021

Thanks!

GoeLin
GoeLin approved these changes Oct 29, 2021
Copy link
Member

@GoeLin GoeLin left a comment

LGTM

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Oct 29, 2021

LGTM

Thanks Goetz!

@openjdk
Copy link

@openjdk openjdk bot commented Oct 29, 2021

@tstuefe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8276086: Increase size of metaspace mappings

Reviewed-by: goetz, mdoerr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 23 new commits pushed to the master branch:

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Oct 29, 2021
Copy link
Contributor

@TheRealMDoerr TheRealMDoerr left a comment

Looks good to me, too.

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Oct 29, 2021

Thank you @GoeLin and @TheRealMDoerr !

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Oct 29, 2021

Going to push as commit e89b2c0.
Since your change was applied there have been 23 commits pushed to the master branch:

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Oct 29, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Oct 29, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Oct 29, 2021

@tstuefe Pushed as commit e89b2c0.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@tstuefe tstuefe deleted the JDK-8276086-increase-metaspace-mapping-size branch Nov 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime integrated
3 participants