Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8254799: runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java fails with release VMs #673

Closed
wants to merge 2 commits into from

Conversation

DamonFool
Copy link
Member

@DamonFool DamonFool commented Oct 14, 2020

runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java and runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java fail with release VMs due to VerifyDependencies is develop and is available only in debug version of VM.
-XX:+IgnoreUnrecognizedVMOptions is added to fix it.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Testing

Linux x64 Windows x64 macOS x64
Build ✔️ (5/5 passed) ✔️ (2/2 passed) ✔️ (2/2 passed)
Test (tier1) ✔️ (9/9 passed) ✔️ (9/9 passed) ✔️ (9/9 passed)

Issue

  • JDK-8254799: runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java fails with release VMs

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/673/head:pull/673
$ git checkout pull/673

@DamonFool
Copy link
Member Author

@DamonFool DamonFool commented Oct 14, 2020

/issue add JDK-8254799
/label add hotspot-runtime
/cc hotspot-runtime

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Oct 14, 2020

👋 Welcome back jiefu! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Oct 14, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Oct 14, 2020

@DamonFool This issue is referenced in the PR title - it will now be updated.

@openjdk openjdk bot added the hotspot-runtime label Oct 14, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Oct 14, 2020

@DamonFool
The hotspot-runtime label was successfully added.

@openjdk
Copy link

@openjdk openjdk bot commented Oct 14, 2020

@DamonFool The hotspot-runtime label was already applied.

@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 14, 2020

Webrevs

@yminqi
Copy link
Contributor

@yminqi yminqi commented Oct 15, 2020

For test requires debug version, please use "@requires vm.debug" in test.

Copy link
Member

@dholmes-ora dholmes-ora left a comment

Looks good and trivial.
Thanks for fixing.

David

@openjdk
Copy link

@openjdk openjdk bot commented Oct 15, 2020

@DamonFool This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254799: runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java fails with release VMs

Reviewed-by: dholmes, dcubed, stuefe

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 13 new commits pushed to the master branch:

  • cda22e3: 8254811: JDK-8254158 broke ppc64, s390 builds
  • 8ef88e5: 8249878: jfr_emergency_dump has secondary crashes
  • 3c2f5e0: 8254674: G1: Improve root location reference to dead obj verification message
  • abe5137: 8254719: ZGC: Clean up includes
  • 60159cf: 8253952: Refine ZipOutputStream.putNextEntry() to recalculate ZipEntry's compressed size
  • 9359ff0: 8252221: Use multiple workers for Parallel GC pre-touching
  • f44fc6d: 8254734: "dead loop detected" assert failure with patch from 8223051
  • 7f73474: 8254773: Remove unimplemented ciReplay::is_loaded(Klass* klass)
  • 167c192: 8254771: Remove unimplemented ciSignature::get_all_klasses
  • 81a8ff1: 8254769: Remove unimplemented BCEscapeAnalyzer::{add_dependence, propagate_dependencies}
  • ... and 3 more: https://git.openjdk.java.net/jdk/compare/55d760d416e6f4c1e03d5101722567c2ffb53b3d...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Oct 15, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 15, 2020

Mailing list message from David Holmes on hotspot-runtime-dev:

Hi Yumin,

On 15/10/2020 10:16 am, Yumin Qi wrote:

On Wed, 14 Oct 2020 23:45:52 GMT, Jie Fu <jiefu at openjdk.org> wrote:

runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java and
runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java fail with release VMs due to VerifyDependencies is
develop and is available only in debug version of VM.
-XX:+IgnoreUnrecognizedVMOptions is added to fix it.

For test using debug version, please use "@requires vm.debug" in test.

I disagree with that approach. The test is run for any release and this
is exec'ing the JVM directly so not controlled by an @test directive
directly. You would have to have two versions of the Java code to run on
release or debug VMs.

What Jie has done seems the best solution to me - a logical equivalent of

DEBUG_ONLY(-XX:-VerifyDependencies)

Cheers,
David

@yminqi
Copy link
Contributor

@yminqi yminqi commented Oct 15, 2020

Mailing list message from David Holmes on hotspot-runtime-dev:

Hi Yumin,

On 15/10/2020 10:16 am, Yumin Qi wrote:

On Wed, 14 Oct 2020 23:45:52 GMT, Jie Fu wrote:

runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java and
runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java fail with release VMs due to VerifyDependencies is
develop and is available only in debug version of VM.
-XX:+IgnoreUnrecognizedVMOptions is added to fix it.

For test using debug version, please use "@requires vm.debug" in test.

I disagree with that approach. The test is run for any release and this
is exec'ing the JVM directly so not controlled by an @test directive
directly. You would have to have two versions of the Java code to run on
release or debug VMs.

What Jie has done seems the best solution to me - a logical equivalent of

DEBUG_ONLY(-XX:-VerifyDependencies)

Cheers,
David

The original fix has problem, don't know how it went through test! Disabling this flag is to reduce run time in debug. I am OK with the workaround though it looks strange.

@yminqi
Copy link
Contributor

@yminqi yminqi commented Oct 15, 2020

Mailing list message from David Holmes on hotspot-runtime-dev:
Hi Yumin,
On 15/10/2020 10:16 am, Yumin Qi wrote:

On Wed, 14 Oct 2020 23:45:52 GMT, Jie Fu wrote:

runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java and
runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java fail with release VMs due to VerifyDependencies is
develop and is available only in debug version of VM.
-XX:+IgnoreUnrecognizedVMOptions is added to fix it.

For test using debug version, please use "@requires vm.debug" in test.

I disagree with that approach. The test is run for any release and this
is exec'ing the JVM directly so not controlled by an @test directive
directly. You would have to have two versions of the Java code to run on
release or debug VMs.
What Jie has done seems the best solution to me - a logical equivalent of
DEBUG_ONLY(-XX:-VerifyDependencies)
Cheers,
David

The original fix has problem, don't know how it went through test! Disabling this flag is to reduce run time in debug. I am OK with the workaround though it looks strange.

Maybe use Platform.isDebugBuild()? Anyway up to Jie's choice.

"-XX:+IgnoreUnrecognizedVMOptions",
"-XX:-VerifyDependencies",
Copy link
Member

@dcubed-ojdk dcubed-ojdk Oct 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding "-XX:+IgnoreUnrecognizedVMOptions", you can
check the JDK type like this:

String jdkType = System.getProperty("jdk.debug", "release");
boolean addNonReleaseOptions = false;
if (!jdkType.equals("release")) {
    addNonReleaseOptions = true;
}

and then only include the "-XX:-VerifyDependencies" option
when addNonReleaseOptions is true... I'm not sure how to
do optional parameters with ProcessTools.createJavaProcessBuilder().

Copy link
Member

@dcubed-ojdk dcubed-ojdk Oct 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know about Platform.isDebugBuild()...

@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 15, 2020

Mailing list message from David Holmes on hotspot-runtime-dev:

On 15/10/2020 10:48 am, Daniel D.Daugherty wrote:

On Wed, 14 Oct 2020 23:45:52 GMT, Jie Fu <jiefu at openjdk.org> wrote:

runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java and
runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java fail with release VMs due to VerifyDependencies is
develop and is available only in debug version of VM.
-XX:+IgnoreUnrecognizedVMOptions is added to fix it.

Changes requested by dcubed (Reviewer).

test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java line 82:

80: "-XX:MaxMetaspaceSize=16m",
81: "-XX:+IgnoreUnrecognizedVMOptions",
82: "-XX:-VerifyDependencies",

Instead of adding "-XX:+IgnoreUnrecognizedVMOptions", you can
check the JDK type like this:

String jdkType = System.getProperty("jdk.debug", "release");
boolean addNonReleaseOptions = false;
if (!jdkType.equals("release")) {
addNonReleaseOptions = true;
}

and then only include the "-XX:-VerifyDependencies" option
when `addNonReleaseOptions` is true... I'm not sure how to
do optional parameters with ProcessTools.createJavaProcessBuilder().

I think this is a trivial issue being over engineered. :) We run many
tests with -XX:+IgnoreUnrecognizedVMOptions at the jtreg level or the
@run level, and this is no different.

Cheers,
David
-----

@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 15, 2020

Mailing list message from Daniel D. Daugherty on hotspot-runtime-dev:

On 10/14/20 8:59 PM, David Holmes wrote:

On 15/10/2020 10:48 am, Daniel D.Daugherty wrote:

On Wed, 14 Oct 2020 23:45:52 GMT, Jie Fu <jiefu at openjdk.org> wrote:

runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java and
runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java
fail with release VMs due to VerifyDependencies is
develop and is available only in debug version of VM.
-XX:+IgnoreUnrecognizedVMOptions is added to fix it.

Changes requested by dcubed (Reviewer).

test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java
line 82:

80: "-XX:MaxMetaspaceSize=16m",
81:???????????????? "-XX:+IgnoreUnrecognizedVMOptions",
82:???????????????? "-XX:-VerifyDependencies",

Instead of adding "-XX:+IgnoreUnrecognizedVMOptions", you can
check the JDK type like this:

String jdkType = System.getProperty("jdk.debug", "release");
boolean addNonReleaseOptions = false;
if (!jdkType.equals("release")) {
???? addNonReleaseOptions = true;
}

and then only include the "-XX:-VerifyDependencies" option
when `addNonReleaseOptions` is true... I'm not sure how to
do optional parameters with ProcessTools.createJavaProcessBuilder().

I think this is a trivial issue being over engineered. :) We run many
tests with -XX:+IgnoreUnrecognizedVMOptions at the jtreg level or the
@run level, and this is no different.

My understanding is that we are trying to stop using
-XX:+IgnoreUnrecognizedVMOptions because it has unexpected
side effects.

Dan

Cheers,
David
-----

@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 15, 2020

Mailing list message from David Holmes on hotspot-runtime-dev:

On 15/10/2020 11:01 am, Daniel D. Daugherty wrote:

On 10/14/20 8:59 PM, David Holmes wrote:

On 15/10/2020 10:48 am, Daniel D.Daugherty wrote:

On Wed, 14 Oct 2020 23:45:52 GMT, Jie Fu <jiefu at openjdk.org> wrote:

runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java and
runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java
fail with release VMs due to VerifyDependencies is
develop and is available only in debug version of VM.
-XX:+IgnoreUnrecognizedVMOptions is added to fix it.

Changes requested by dcubed (Reviewer).

test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java
line 82:

80: "-XX:MaxMetaspaceSize=16m",
81:???????????????? "-XX:+IgnoreUnrecognizedVMOptions",
82:???????????????? "-XX:-VerifyDependencies",

Instead of adding "-XX:+IgnoreUnrecognizedVMOptions", you can
check the JDK type like this:

String jdkType = System.getProperty("jdk.debug", "release");
boolean addNonReleaseOptions = false;
if (!jdkType.equals("release")) {
???? addNonReleaseOptions = true;
}

and then only include the "-XX:-VerifyDependencies" option
when `addNonReleaseOptions` is true... I'm not sure how to
do optional parameters with ProcessTools.createJavaProcessBuilder().

I think this is a trivial issue being over engineered. :) We run many
tests with -XX:+IgnoreUnrecognizedVMOptions at the jtreg level or the
@run level, and this is no different.

My understanding is that we are trying to stop using
-XX:+IgnoreUnrecognizedVMOptions because it has unexpected
side effects.

I don't recall that.

But in this case it is not an issue at all. We are using
ProcessTools.createJavaProcessBuilder to pass the specific set of flags
listed in the test itself. There are no incoming flags from jtreg etc,
so nothing that can be affected. Further, even if using createTestJVM,
the explicit flags are placed last on the command-line so still cannot
interfere with anything coming in from jtreg.

I suppose a trivial change for Jie would be:

- "-XX:-VerifyDependencies",
+ Platform.isDebugBuild() ? "-XX:-VerifyDependencies" : "",

Cheers,
David

Dan

Cheers,
David
-----

@DamonFool
Copy link
Member Author

@DamonFool DamonFool commented Oct 15, 2020

I like @dholmes-ora 's solution and will update the PR soon.
Thanks.

@DamonFool
Copy link
Member Author

@DamonFool DamonFool commented Oct 15, 2020

Unfortunately, the following fix doesn't work.

Platform.isDebugBuild() ? "-XX:-VerifyDependencies" : "",

It seems that ProcessTools.createJavaProcessBuilder doesn't allow empty string as an arg.

Would you mind something like:

Platform.isDebugBuild() ? "-XX:-VerifyDependencies" : "-XX:MaxMetaspaceSize=16m",

If you don't like it, I still prefer -XX:+IgnoreUnrecognizedVMOptions.
I've grepped under jdk/test finding that there are about 700+ tests using -XX:+IgnoreUnrecognizedVMOptions.
I didn't get the point why -XX:+IgnoreUnrecognizedVMOptions should be avoided for this case.

@mlbridge
Copy link

@mlbridge mlbridge bot commented Oct 15, 2020

Mailing list message from David Holmes on hotspot-runtime-dev:

On 15/10/2020 12:06 pm, Jie Fu wrote:

On Thu, 15 Oct 2020 01:24:21 GMT, Jie Fu <jiefu at openjdk.org> wrote:

Changes requested by dcubed (Reviewer).

I like @dholmes-ora 's solution and will update the PR soon.
Thanks.

Unfortunately, the following fix doesn't work.
Platform.isDebugBuild() ? "-XX:-VerifyDependencies" : "",
It seems that ProcessTools.createJavaProcessBuilder doesn't allow empty string as an arg.

Well that is annoying - not quite sure how that arises but anyway ...

Would you mind something like:
Platform.isDebugBuild() ? "-XX:-VerifyDependencies" : "-XX:MaxMetaspaceSize=16m",

Just use "-Dx" as the dummy argument.

Thanks,
David

@yminqi
Copy link
Contributor

@yminqi yminqi commented Oct 15, 2020

Hi, Jie, David and Dan, I would like to reorg the code a little bit:

`index a889ae11ed1..596ca348c30 100644
--- a/test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java
+++ b/test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java
@@ -31,10 +31,12 @@
import jdk.test.lib.Asserts;
import jdk.test.lib.classloader.GeneratingClassLoader;
import jdk.test.lib.hprof.HprofParser;
+import jdk.test.lib.Platform;
import jdk.test.lib.process.ProcessTools;
import jdk.test.lib.process.OutputAnalyzer;

import java.io.File;
+import java.util.ArrayList;

public class TestHeapDumpOnOutOfMemoryError {

@@ -64,22 +66,29 @@ public class TestHeapDumpOnOutOfMemoryError {
}

 static void test(String type) throws Exception {
  •    ArrayList<String> args = new ArrayList();
    
  •    String heapdumpFilename = type + ".hprof";
    
  •    ProcessBuilder pb = ProcessTools.createJavaProcessBuilder("-XX:+HeapDumpOnOutOfMemoryError",
    
  •            "-XX:HeapDumpPath=" + heapdumpFilename,
    
  •            // Note: When trying to provoke a metaspace OOM we may generate a lot of classes. In debug VMs this
    
  •            //  can cause considerable wait times since:
    
  •            // - Compiler Dependencies verification iterates the class tree
    
  •            // - Before exit, the CLDG is checked.
    
  •            // Both verifications show quadratic time or worse wrt to number of loaded classes. Therefore it
    
  •            //  makes sense to switch one or both off and limit the metaspace size to something sensible.
    
  •            // Example numbers on a slow ppc64 machine:
    
  •            //  MaxMetaspaceSize=64M - ~60-70K classes - ~20min runtime with all verifications
    
  •            //  MaxMetaspaceSize=16M - ~12-15K classes - ~12sec runtime with all verifications
    
  •            //  MaxMetaspaceSize=16M - ~12-15K classes - VerifyDependencies off - ~3seconds on ppc
    
  •            "-XX:MaxMetaspaceSize=16m",
    
  •            "-XX:-VerifyDependencies",
    
  •            TestHeapDumpOnOutOfMemoryError.class.getName(), type);
    
  •    // Note: When trying to provoke a metaspace OOM we may generate a lot of classes. In debug VMs this
    
  •    //  can cause considerable wait times since:
    
  •    // - Compiler Dependencies verification iterates the class tree
    
  •    // - Before exit, the CLDG is checked.
    
  •    // Both verifications show quadratic time or worse wrt to number of loaded classes. Therefore it
    
  •    //  makes sense to switch one or both off and limit the metaspace size to something sensible.
    
  •    // Example numbers on a slow ppc64 machine:
    
  •    //  MaxMetaspaceSize=64M - ~60-70K classes - ~20min runtime with all verifications
    
  •    //  MaxMetaspaceSize=16M - ~12-15K classes - ~12sec runtime with all verifications
    
  •    //  MaxMetaspaceSize=16M - ~12-15K classes - VerifyDependencies off - ~3seconds on ppc
    
  •    args.add("-XX:+HeapDumpOnOutOfMemoryError");
    
  •    args.add("-XX:HeapDumpPath=" + heapdumpFilename);
    
  •    args.add("-XX:MaxMetaspaceSize=16m");
    
  •    if (Platform.isDebugBuild()) {
    
  •        args.add("-XX:-VerifyDependencies");
    
  •    }
    
  •    args.add(TestHeapDumpOnOutOfMemoryError.class.getName());
    
  •    args.add(type);
    
  •    ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(args);
    
       OutputAnalyzer output = new OutputAnalyzer(pb.start());
       output.stdoutShouldNotBeEmpty();`
    

What do you think?

@yminqi
Copy link
Contributor

@yminqi yminqi commented Oct 15, 2020

Aha, the diff looks ugly. Basically ProcessTools.createJavaProcessBuilder(List<?>) so you can manage the code there.

@DamonFool
Copy link
Member Author

@DamonFool DamonFool commented Oct 15, 2020

Hi @yminqi
To fix this bug, I prefer David's suggestion, which seems simple and elegant.
Thanks.

@DamonFool
Copy link
Member Author

@DamonFool DamonFool commented Oct 15, 2020

Hi all,

I prefer David's solution because I think it's a good idea to avoid -XX:+IgnoreUnrecognizedVMOptions without the effort to reorg the code.
I like this approach if -XX:+IgnoreUnrecognizedVMOptions is not allowed.

I've updated the PR.
Please review it.
Thanks.

Best regards,
Jie

Copy link
Member

@tstuefe tstuefe left a comment

Sorry for the trouble my change caused. :(

It would have been fine also to just run this test on debug only, or even just to remove the option; the speedup that the reduced MaxMetaspaceSize brought would have been enough.

Thanks a lot for fixing this.

I wondered how it got past gatekeeper tests but I see now that it was excluded from tier1, probably exactly because the slow runtime.

@DamonFool
Copy link
Member Author

@DamonFool DamonFool commented Oct 15, 2020

Hi @dcubed-ojdk ,
Hope you have no objection to the latest fix.
And then I can push it ASAP.
Thanks.

Copy link
Member

@dcubed-ojdk dcubed-ojdk left a comment

Thumbs up on this version.
I would have used "-Dignored_option" so it was self documenting,
but that's me... "-Dx" works just fine.

@DamonFool
Copy link
Member Author

@DamonFool DamonFool commented Oct 15, 2020

Thanks @yminqi , @dholmes-ora , @dcubed-ojdk and @tstuefe for your helpful comments and review.
/integrate

@openjdk openjdk bot closed this Oct 15, 2020
@openjdk openjdk bot added integrated and removed ready rfr labels Oct 15, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Oct 15, 2020

@DamonFool Since your change was applied there have been 13 commits pushed to the master branch:

  • cda22e3: 8254811: JDK-8254158 broke ppc64, s390 builds
  • 8ef88e5: 8249878: jfr_emergency_dump has secondary crashes
  • 3c2f5e0: 8254674: G1: Improve root location reference to dead obj verification message
  • abe5137: 8254719: ZGC: Clean up includes
  • 60159cf: 8253952: Refine ZipOutputStream.putNextEntry() to recalculate ZipEntry's compressed size
  • 9359ff0: 8252221: Use multiple workers for Parallel GC pre-touching
  • f44fc6d: 8254734: "dead loop detected" assert failure with patch from 8223051
  • 7f73474: 8254773: Remove unimplemented ciReplay::is_loaded(Klass* klass)
  • 167c192: 8254771: Remove unimplemented ciSignature::get_all_klasses
  • 81a8ff1: 8254769: Remove unimplemented BCEscapeAnalyzer::{add_dependence, propagate_dependencies}
  • ... and 3 more: https://git.openjdk.java.net/jdk/compare/55d760d416e6f4c1e03d5101722567c2ffb53b3d...master

Your commit was automatically rebased without conflicts.

Pushed as commit f3ce45f.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@DamonFool DamonFool deleted the JDK-8254799 branch Oct 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime integrated
5 participants