Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8283422: Create a new test for JDK-8254790 #7883

Closed
wants to merge 8 commits into from

Conversation

mrserb
Copy link
Member

@mrserb mrserb commented Mar 21, 2022

The JDK-8254790 initially was reported on macOS.
It was found by the "javax/xml/crypto/dsig/GenerationTests.java" test which for some reason worked fine on Linux/Windows most? of the time.

I tried to run that test on Linux/Windows many times on a few systems and it always pass. Also, I got the same result for the tier1...tier4 tests.

I have found that the easiest way to reproduce the crash is to pass the result of the string.indexOf to the str.charAt when the heap is big enough. The test below[1] trigger the crash 1000 time in a row on win/lin on systems I have tested.

I am not sure what is the best way to contribute to the test, should I create a stand-alone test like[1], or update an existing test as I did for the first version of this PR?

[1]:

package compiler.intrinsics.string;

import static jdk.test.lib.Asserts.assertTrue;

public final class TestStringIndexOfLargeHeap {
    static final int[] data = new int[Integer.MAX_VALUE / 2];
    public static void main(String[] args) throws Exception {
        test('a');
        test('\ubeef');
    }
    private static void test(char ch) {
        String str = ("0123456789" + ch).repeat(100);
        for (int i = 0; i < 1_000_000; ++i) {
            assertTrue(ch == str.charAt(str.indexOf(ch)));
        }
    }
}

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7883/head:pull/7883
$ git checkout pull/7883

Update a local copy of the PR:
$ git checkout pull/7883
$ git pull https://git.openjdk.java.net/jdk pull/7883/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7883

View PR using the GUI difftool:
$ git pr show -t 7883

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7883.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Mar 21, 2022

👋 Welcome back serb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Mar 21, 2022

@mrserb The following label will be automatically applied to this pull request:

  • hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-compiler hotspot-compiler-dev@openjdk.org label Mar 21, 2022
@@ -43,6 +43,28 @@
* compiler.intrinsics.string.TestStringIntrinsics2
*/

/*
* @test id=LargeHeap
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it useful to run all other tests in the TestStringIntrinsics2 using the large heap?

@mrserb mrserb marked this pull request as ready for review March 21, 2022 09:07
@openjdk openjdk bot added the rfr Pull request is ready for review label Mar 21, 2022
@mlbridge
Copy link

mlbridge bot commented Mar 21, 2022

Webrevs

@TobiHartmann
Copy link
Member

The test fails in our CI with:

java.lang.OutOfMemoryError: Java heap space
	at compiler.intrinsics.string.TestStringIntrinsics2.main(TestStringIntrinsics2.java:142)
	at java.base/java.lang.invoke.LambdaForm$DMH/0x0000000801002800.invokeStatic(LambdaForm$DMH)
	at java.base/java.lang.invoke.LambdaForm$MH/0x0000000801003c00.invoke(LambdaForm$MH)
	at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:155)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
	at java.base/java.lang.Thread.run(Thread.java:828)

@mrserb
Copy link
Member Author

mrserb commented Mar 22, 2022

@TobiHartmann Thank you for your feedback, the trace shows that the test fails to allocate the FILL_HEAP = new int[Integer.MAX_VALUE / 2]; when the heap is set to 5G. I can increase the "-mx" option, but probably the root cause of the OOM is something different, the "git actions" are green, the test is also green when I run it multiple times. Probably mach5 adds some additional options to the test execution?

@TobiHartmann
Copy link
Member

I think the root cause is simply that there is not enough memory available to the VM to allocate such a large object. MaxRAMPercentage is set to 4 but the test could also be executed on a system with a small memory size. I think the test needs to be robust enough to account for all kinds of machines/settings.

Do you know why the issue is only reproducible with a large heap and only if the heap is filled?

@mrserb
Copy link
Member Author

mrserb commented Mar 23, 2022

I think the root cause is simply that there is not enough memory available to the VM to allocate such a large object. MaxRAMPercentage is set to 4 but the test could also be executed on a system with a small memory size. I think the test needs to be robust enough to account for all kinds of machines/settings.

It has a "guard" jtreg tag "os.maxMemory > 5G" which should prevent execution of this test on small systems. I'll bump this requirement in the next revision of the test.

Do you know why the issue is only reproducible with a large heap and only if the heap is filled?

That's due to the nature of that bug

the 32 bit arithmetic instruction used for 64-bit address

@mrserb
Copy link
Member Author

mrserb commented Mar 30, 2022

I have found the bug where some tests tried to allocate same huge amount of memory and fail:
https://bugs.openjdk.java.net/browse/JDK-8227645
The solution was to move them to the special folder "hotspot/jtreg/resourcehogs" so they will have "exclusiveAccess" flag.
I'll "move" the new test there.

@mrserb mrserb marked this pull request as draft March 30, 2022 04:25
@openjdk openjdk bot removed the rfr Pull request is ready for review label Mar 30, 2022
@mrserb mrserb marked this pull request as ready for review March 30, 2022 07:45
@openjdk openjdk bot added the rfr Pull request is ready for review label Mar 30, 2022
@mrserb
Copy link
Member Author

mrserb commented Mar 30, 2022

The test is updated:

  • The special wrapper TestStringIntrinsics2LargeArray for the TestStringIntrinsics2 is added to the test/hotspot/jtreg/resourcehogs/. This approach is based on the similar test. Note that the tests in the resourcehogs folder use the exclusiveAccess property.
  • The new test will be executed as part of the hotspot:tier3

Copy link
Contributor

@vnkozlov vnkozlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your proposed test configuration. Let me test it before approval.

Copy link
Contributor

@vnkozlov vnkozlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing results are good.

@openjdk
Copy link

openjdk bot commented Mar 30, 2022

@mrserb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8283422: Create a new test for JDK-8254790

Reviewed-by: kvn, thartmann

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 49 new commits pushed to the master branch:

  • e0a8669: 8281223: Improve the API documentation of HttpRequest.Builder::build to state that the default implementation provided by the JDK returns immutable objects.
  • eeca3a3: 8253569: javax.xml.catalog.Catalog.matchURI() implementation should reset state variables
  • ec0897a: 8281705: SourceLauncherTest.testSystemProperty isn't being run
  • 1ddab6f: 8283060: RawNativeLibraries should allow multiple clients to load/unload the same library
  • 835c7e8: 8283907: Fix Huawei copyright in various files
  • b8dd21b: 8283801: Cleanup confusing String.toString calls
  • ce27d9d: 8283494: Factor out calculation of actual number of XMM registers
  • e61ccfb: 8283470: Update java.lang.invoke.VarHandle to use sealed classes
  • e85fa2f: 8283700: Add final or sealed modifier to appropriate java.awt API classes
  • ec9ab55: 8282374: Java_sun_awt_X11_XlibWrapper_XSynchronize is wrong and unused
  • ... and 39 more: https://git.openjdk.java.net/jdk/compare/d8fd22239bafecdaaedb8985ab6d709ed846e808...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Mar 30, 2022
Copy link
Member

@TobiHartmann TobiHartmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me too.

@mrserb
Copy link
Member Author

mrserb commented Mar 31, 2022

/integrate

@openjdk
Copy link

openjdk bot commented Mar 31, 2022

Going to push as commit ad83ec7.
Since your change was applied there have been 69 commits pushed to the master branch:

  • 6ebf845: 8283566: G1: Improve G1BarrierSet::enqueue performance
  • d276da5: 8281469: aarch64: Improve interpreter stack banging
  • a41550b: 8283842: TestZoneTextPrinterParser.test_roundTripAtOverlap fails: DateTimeParseException
  • 207b099: 8283890: Changes in CFG file format break C1Visualizer
  • 49fcc7a: 8283013: Simplify Arguments::parse_argument()
  • 73cb922: 8284026: Use unmodifiable collections where practical
  • 77a205a: 8284090: com/sun/security/auth/module/AllPlatforms.java fails to compile
  • 64025b0: 8283901: Introduce "make doctor" to diagnose build environment problems
  • 5740a3b: 8280193: summary javadoc for java.awt.GraphicsEnvironment#preferProportionalFonts broken
  • a11cc97: 8283997: Unused argument in GraphKit::builtin_throw
  • ... and 59 more: https://git.openjdk.java.net/jdk/compare/d8fd22239bafecdaaedb8985ab6d709ed846e808...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Mar 31, 2022
@openjdk openjdk bot closed this Mar 31, 2022
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Mar 31, 2022
@openjdk
Copy link

openjdk bot commented Mar 31, 2022

@mrserb Pushed as commit ad83ec7.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@mrserb mrserb deleted the JDK-8283422 branch March 31, 2022 18:19
@openjdk
Copy link

openjdk bot commented Apr 12, 2022

@mrserb Unknown command backport - for a list of valid commands use /help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-compiler hotspot-compiler-dev@openjdk.org integrated Pull request has been integrated
3 participants