8283422: Create a new test for JDK-8254790

[mrserb]

The JDK-8254790 initially was reported on macOS.
It was found by the "javax/xml/crypto/dsig/GenerationTests.java" test which for some reason worked fine on Linux/Windows most? of the time.

I tried to run that test on Linux/Windows many times on a few systems and it always pass. Also, I got the same result for the tier1...tier4 tests.

I have found that the easiest way to reproduce the crash is to pass the result of the string.indexOf to the str.charAt when the heap is big enough. The test below[1] trigger the crash 1000 time in a row on win/lin on systems I have tested.

I am not sure what is the best way to contribute to the test, should I create a stand-alone test like[1], or update an existing test as I did for the first version of this PR?

[1]:

package compiler.intrinsics.string;

import static jdk.test.lib.Asserts.assertTrue;

public final class TestStringIndexOfLargeHeap {
    static final int[] data = new int[Integer.MAX_VALUE / 2];
    public static void main(String[] args) throws Exception {
        test('a');
        test('\ubeef');
    }
    private static void test(char ch) {
        String str = ("0123456789" + ch).repeat(100);
        for (int i = 0; i < 1_000_000; ++i) {
            assertTrue(ch == str.charAt(str.indexOf(ch)));
        }
    }
}

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8283422: Create a new test for JDK-8254790

Reviewers

• Vladimir Kozlov (@vnkozlov - Reviewer)
• Tobias Hartmann (@TobiHartmann - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7883/head:pull/7883
$ git checkout pull/7883

Update a local copy of the PR:
$ git checkout pull/7883
$ git pull https://git.openjdk.java.net/jdk pull/7883/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7883

View PR using the GUI difftool:
$ git pr show -t 7883

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7883.diff

[bridgekeeper]

👋 Welcome back serb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@mrserb The following label will be automatically applied to this pull request:

• hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mrserb]

Is it useful to run all other tests in the TestStringIntrinsics2 using the large heap?

[mlbridge]

Webrevs

• 03: Full - Incremental (feded853)
• 02: Full - Incremental (358813a5)
• 01: Full - Incremental (ab2fbc35)
• 00: Full (78c76a4e)

[TobiHartmann]

The test fails in our CI with:

java.lang.OutOfMemoryError: Java heap space
	at compiler.intrinsics.string.TestStringIntrinsics2.main(TestStringIntrinsics2.java:142)
	at java.base/java.lang.invoke.LambdaForm$DMH/0x0000000801002800.invokeStatic(LambdaForm$DMH)
	at java.base/java.lang.invoke.LambdaForm$MH/0x0000000801003c00.invoke(LambdaForm$MH)
	at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:155)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
	at java.base/java.lang.Thread.run(Thread.java:828)

[mrserb]

@TobiHartmann Thank you for your feedback, the trace shows that the test fails to allocate the FILL_HEAP = new int[Integer.MAX_VALUE / 2]; when the heap is set to 5G. I can increase the "-mx" option, but probably the root cause of the OOM is something different, the "git actions" are green, the test is also green when I run it multiple times. Probably mach5 adds some additional options to the test execution?

[TobiHartmann]

I think the root cause is simply that there is not enough memory available to the VM to allocate such a large object. MaxRAMPercentage is set to 4 but the test could also be executed on a system with a small memory size. I think the test needs to be robust enough to account for all kinds of machines/settings.

Do you know why the issue is only reproducible with a large heap and only if the heap is filled?

[mrserb]

I think the root cause is simply that there is not enough memory available to the VM to allocate such a large object. MaxRAMPercentage is set to 4 but the test could also be executed on a system with a small memory size. I think the test needs to be robust enough to account for all kinds of machines/settings.

It has a "guard" jtreg tag "os.maxMemory > 5G" which should prevent execution of this test on small systems. I'll bump this requirement in the next revision of the test.

Do you know why the issue is only reproducible with a large heap and only if the heap is filled?

That's due to the nature of that bug

the 32 bit arithmetic instruction used for 64-bit address

[mrserb]

I have found the bug where some tests tried to allocate same huge amount of memory and fail:
https://bugs.openjdk.java.net/browse/JDK-8227645
The solution was to move them to the special folder "hotspot/jtreg/resourcehogs" so they will have "exclusiveAccess" flag.
I'll "move" the new test there.

[mrserb]

The test is updated:

• The special wrapper TestStringIntrinsics2LargeArray for the TestStringIntrinsics2 is added to the test/hotspot/jtreg/resourcehogs/. This approach is based on the similar test. Note that the tests in the resourcehogs folder use the exclusiveAccess property.
• The new test will be executed as part of the hotspot:tier3

[vnkozlov]

I agree with your proposed test configuration. Let me test it before approval.

[vnkozlov]

Testing results are good.

[openjdk]

@mrserb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8283422: Create a new test for JDK-8254790

Reviewed-by: kvn, thartmann

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 49 new commits pushed to the master branch:

• e0a8669: 8281223: Improve the API documentation of HttpRequest.Builder::build to state that the default implementation provided by the JDK returns immutable objects.
• eeca3a3: 8253569: javax.xml.catalog.Catalog.matchURI() implementation should reset state variables
• ec0897a: 8281705: SourceLauncherTest.testSystemProperty isn't being run
• 1ddab6f: 8283060: RawNativeLibraries should allow multiple clients to load/unload the same library
• 835c7e8: 8283907: Fix Huawei copyright in various files
• b8dd21b: 8283801: Cleanup confusing String.toString calls
• ce27d9d: 8283494: Factor out calculation of actual number of XMM registers
• e61ccfb: 8283470: Update java.lang.invoke.VarHandle to use sealed classes
• e85fa2f: 8283700: Add final or sealed modifier to appropriate java.awt API classes
• ec9ab55: 8282374: Java_sun_awt_X11_XlibWrapper_XSynchronize is wrong and unused
• ... and 39 more: https://git.openjdk.java.net/jdk/compare/d8fd22239bafecdaaedb8985ab6d709ed846e808...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[TobiHartmann]

Looks good to me too.

[mrserb]

/integrate

[openjdk]

Going to push as commit ad83ec7.
Since your change was applied there have been 69 commits pushed to the master branch:

• 6ebf845: 8283566: G1: Improve G1BarrierSet::enqueue performance
• d276da5: 8281469: aarch64: Improve interpreter stack banging
• a41550b: 8283842: TestZoneTextPrinterParser.test_roundTripAtOverlap fails: DateTimeParseException
• 207b099: 8283890: Changes in CFG file format break C1Visualizer
• 49fcc7a: 8283013: Simplify Arguments::parse_argument()
• 73cb922: 8284026: Use unmodifiable collections where practical
• 77a205a: 8284090: com/sun/security/auth/module/AllPlatforms.java fails to compile
• 64025b0: 8283901: Introduce "make doctor" to diagnose build environment problems
• 5740a3b: 8280193: summary javadoc for java.awt.GraphicsEnvironment#preferProportionalFonts broken
• a11cc97: 8283997: Unused argument in GraphKit::builtin_throw
• ... and 59 more: https://git.openjdk.java.net/jdk/compare/d8fd22239bafecdaaedb8985ab6d709ed846e808...master

Your commit was automatically rebased without conflicts.

[openjdk]

@mrserb Pushed as commit ad83ec7.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

[openjdk]

@mrserb Unknown command backport - for a list of valid commands use /help.