Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8315770: serviceability/sa/TestJmapCoreMetaspace.java should run with -XX:-VerifyDependencies #15631

Closed

Conversation

roy-soumadipta
Copy link
Contributor

@roy-soumadipta roy-soumadipta commented Sep 8, 2023

8315770: serviceability/sa/TestJmapCoreMetaspace.java should run with -XX:-VerifyDependencies

serviceability/sa/TestJmapCoreMetaspace.java runs in hotspot:tier2, and takes about 330 seconds out of 670 seconds of the entire run on x86_64 fastdebug. The tier2 completion usually waits on it. Profiling shows we are spending time during classloading, checking dependencies. Since we load lots of classes, we do it many times, and each time the cost grows linearly to the number of classes. Some other tests that target metaspace explicitly disable VerifyDependencies to avoid this.

The commit includes adding "-XX:-VerifyDependencies" along with "-XX:+IgnoreUnrecognizedVMOptions" in TestJmapCore.java to make sure that release builds work with the test, as VerifyDependencies is a develop option and not available in release builds. The changes in the commit are resulting in considerable improvement for the test in fastdebug mode as can be seen below:

  • before: 130.41s user 14.11s system 118% cpu 2:01.85 total
  • after: 27.52s user 13.64s system 213% cpu 19.249 total

Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8315770: serviceability/sa/TestJmapCoreMetaspace.java should run with -XX:-VerifyDependencies (Enhancement - P4)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15631/head:pull/15631
$ git checkout pull/15631

Update a local copy of the PR:
$ git checkout pull/15631
$ git pull https://git.openjdk.org/jdk.git pull/15631/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15631

View PR using the GUI difftool:
$ git pr show -t 15631

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15631.diff

Webrev

Link to Webrev Comment

… -XX:-VerifyDependencies

serviceability/sa/TestJmapCoreMetaspace.java runs in hotspot:tier2, and takes about 330 seconds out of 670 seconds of the entire run on x86_64 fastdebug. The tier2 completion usually waits on it. Profiling shows we are spending time during classloading, checking dependencies. Since we load lots of classes, we do it many times, and each time the cost grows linearly to the number of classes. Some other tests that target metaspace explicitly disable `VerifyDependencies` to avoid this.

The commit includes adding "-XX:-VerifyDependencies" along with "-XX:+IgnoreUnrecognizedVMOptions" in TestJmapCore.java to make sure that release builds work with the test, as VerifyDependencies is a develop option and not available in release builds. The changes in the commit is resulting in considerable improvement for the test in fastdebug mode as can be seen below:

* before: **130.41s user 14.11s system 118% cpu 2:01.85 total**
* after: **27.52s user 13.64s system 213% cpu 19.249 total**
@bridgekeeper
Copy link

bridgekeeper bot commented Sep 8, 2023

👋 Welcome back roy-soumadipta! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Sep 8, 2023
@openjdk
Copy link

openjdk bot commented Sep 8, 2023

@roy-soumadipta The following label will be automatically applied to this pull request:

  • serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the serviceability serviceability-dev@openjdk.org label Sep 8, 2023
@mlbridge
Copy link

mlbridge bot commented Sep 8, 2023

Webrevs

@@ -73,6 +73,7 @@ public static void main(String[] args) throws Throwable {
static void test(String type) throws Throwable {
ProcessBuilder pb = ProcessTools.createTestJvm("-XX:+CreateCoredumpOnCrash",
"-Xmx512m", "-XX:MaxMetaspaceSize=64m", "-XX:+CrashOnOutOfMemoryError",
"-XX:-VerifyDependencies", "-XX:+IgnoreUnrecognizedVMOptions",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried this test with release? I think IgnoreUnrecognizedVMOptions is positional, and should go before VerifyDependencies to actually work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try it and raise another commit if that is required

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would make sense to change the order, even if it works right now. This would specify what option is being ignored more clearly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Aleksey, release is working as well

Copy link
Member

@shipilev shipilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Let's ask @plummercj to take a look as well.

@openjdk
Copy link

openjdk bot commented Sep 8, 2023

@roy-soumadipta This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315770: serviceability/sa/TestJmapCoreMetaspace.java should run with -XX:-VerifyDependencies

Reviewed-by: shade, kevinw, cjplummer, lmesnik

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 58 new commits pushed to the master branch:

  • d06a564: 8315765: G1: Incorrect use of G1LastPLABAverageOccupancy
  • 66b6a5a: 8277954: Replace use of monitors with explicit locks in the JDK LDAP provider implementation
  • 4cb4637: 8315970: Big-endian issues after JDK-8310929
  • ae08143: 8315611: Open source swing text/html and tree test
  • 7b3e697: 8315855: G1: Revise signature of set_humongous_candidate
  • 1941290: 8315942: Sort platform enums and definitions after JDK-8304913 follow-ups
  • 996b336: 8315781: Reduce the max value of GCDrainStackTargetSize
  • 35bccac: 8315841: RISC-V: Check for hardware TSO support
  • a04c6c1: 8315609: Open source few more swing text/html tests
  • dab1c21: 8314491: Linux: jexec launched via PATH fails to find java
  • ... and 48 more: https://git.openjdk.org/jdk/compare/024133b089d911dcc3ea70dfdaa6b150b14a9eb4...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@shipilev, @kevinjwalls, @plummercj, @lmesnik) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Sep 8, 2023
Copy link
Contributor

@kevinjwalls kevinjwalls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
I also assumed IgnoreUnrecognizedVMOptions would be positional, but it isn't. I checked various major Java versions and it works after an unknown option.
But I'm very pleased to see the update to have it appear first in the list, it just reads easier and removes that question.

Copy link
Contributor

@plummercj plummercj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good, but I'd suggest adding a comment so the reader knows why these options were added.

Copy link
Member

@lmesnik lmesnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be check if it is a debug build with 'Platform.isDebugBuild()' instead of IgnoreUnrecognizedVMOptions.
If test takes ~30 seconds it does it makes sense to reduce it's timeout to default and return it into tier1 group?

@shipilev
Copy link
Member

The changes look good, but I'd suggest adding a comment so the reader knows why these options were added.

Agreed. Something like:

                "-Xmx512m", "-XX:MaxMetaspaceSize=64m", "-XX:+CrashOnOutOfMemoryError",
                // The test loads lots of small classes to exhaust Metaspace, skip method dependency
                // verification to improve performance in debug builds.
                "-XX:+IgnoreUnrecognizedVMOptions", "-XX:-VerifyDependencies",
                CoreUtils.getAlwaysPretouchArg(true),

@roy-soumadipta
Copy link
Contributor Author

I'm adding a small comment @plummercj and 'Platform.isDebugBuild()' instead of IgnoreUnrecognizedVMOptions @lmesnik in the next commit. But I would like to defer the move to tier1 in a separate commit

…add comment about the change forbetter readability.
Copy link
Member

@shipilev shipilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still good.

@roy-soumadipta
Copy link
Contributor Author

/integrate

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Sep 11, 2023
@openjdk
Copy link

openjdk bot commented Sep 11, 2023

@roy-soumadipta
Your change (at version 7b6af09) is now ready to be sponsored by a Committer.

@shipilev
Copy link
Member

/sponsor

@openjdk
Copy link

openjdk bot commented Sep 11, 2023

Going to push as commit 877731d.
Since your change was applied there have been 58 commits pushed to the master branch:

  • d06a564: 8315765: G1: Incorrect use of G1LastPLABAverageOccupancy
  • 66b6a5a: 8277954: Replace use of monitors with explicit locks in the JDK LDAP provider implementation
  • 4cb4637: 8315970: Big-endian issues after JDK-8310929
  • ae08143: 8315611: Open source swing text/html and tree test
  • 7b3e697: 8315855: G1: Revise signature of set_humongous_candidate
  • 1941290: 8315942: Sort platform enums and definitions after JDK-8304913 follow-ups
  • 996b336: 8315781: Reduce the max value of GCDrainStackTargetSize
  • 35bccac: 8315841: RISC-V: Check for hardware TSO support
  • a04c6c1: 8315609: Open source few more swing text/html tests
  • dab1c21: 8314491: Linux: jexec launched via PATH fails to find java
  • ... and 48 more: https://git.openjdk.org/jdk/compare/024133b089d911dcc3ea70dfdaa6b150b14a9eb4...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Sep 11, 2023
@openjdk openjdk bot closed this Sep 11, 2023
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 11, 2023
@openjdk openjdk bot removed the sponsor Pull request is ready to be sponsored label Sep 11, 2023
@openjdk
Copy link

openjdk bot commented Sep 11, 2023

@shipilev @roy-soumadipta Pushed as commit 877731d.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@roy-soumadipta roy-soumadipta deleted the TestJmapCoreMetaspace branch September 11, 2023 16:47
roy-soumadipta added a commit to roy-soumadipta/jdk21u that referenced this pull request Sep 25, 2023
Backporting the fix for https://bugs.openjdk.org/browse/JDK-8315683 merged as part of openjdk/jdk#15631. https://github.com/openjdk/jdk/commit/877731d2a20249ce4724a071ba2da1faa56daca4.patch has been cleanly applied.

Below are the test results:
* after_release: **54.64s user 7.50s system 231% cpu 26.806 total**
* after_fastdebug: **247.97s user 12.78s system 407% cpu 1:03.99 total**
* before_release: **52.23s user 8.39s system 232% cpu 26.070 total**
* before_fastdebug: **504.20s user 9.34s system 211% cpu 4:03.01 total**
roy-soumadipta added a commit to roy-soumadipta/jdk17u-dev that referenced this pull request Sep 25, 2023
Backporting the fix for https://bugs.openjdk.org/browse/JDK-8315770 merged as part of openjdk/jdk#15631. https://github.com/openjdk/jdk/commit/877731d2a20249ce4724a071ba2da1faa56daca4.patch apply failed due to differences in the file and the changes have been performed selectively.

Below are the test results:
* after_release: **99.63s user 10.34s system 171% cpu 1:04.25 total**
* after_fastdebug: **433.45s user 17.35s system 401% cpu 1:52.15 total**
* before_release: **102.65s user 10.50s system 172% cpu 1:05.65 total**
* before_fastdebug: **516.82s user 16.68s system 253% cpu 3:30.51 total**
roy-soumadipta added a commit to roy-soumadipta/jdk11u-dev that referenced this pull request Sep 27, 2023
Backporting the fix for https://bugs.openjdk.org/browse/JDK-8315770 merged as part of openjdk/jdk#15631. https://github.com/openjdk/jdk/commit/877731d2a20249ce4724a071ba2da1faa56daca4.patch apply failed due to differences in the file and the changes have been performed selectively.

Below are the test results:
* after_release: **123.62s user 10.26s system 160% cpu 1:23.59 total**
* after_fastdebug: **197.42s user 15.07s system 195% cpu 1:48.49 total**
* before_release: **123.65s user 9.56s system 159% cpu 1:23.38 total**
* before_fastdebug: **232.70s user 15.29s system 170% cpu 2:25.70 total**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated serviceability serviceability-dev@openjdk.org
5 participants