8273433: Enable parallelism in vmTestbase_nsk_sysdict tests

[shipilev]

Current vmTestbase_nsk_sysdict suite contains about 20 tests, each running exclusively. There seem to be no reason to run them exclusively, though: they complete in reasonable time, are single-threaded, and consume the usual amount of memory.

We should consider enabling parallelism for them and get improved test performance. Currently it is blocked by TEST.properties with exclusiveAccess.dirs directives in them.

Motivational performance improvements below.

Before:

$ time CONF=linux-x86_64-server-fastdebug make run-test TEST=vmTestbase_nsk_sysdict | ts -s
...
00:45:14 ==============================
00:45:14 Test summary
00:45:14 ==============================
00:45:14    TEST                                              TOTAL  PASS  FAIL ERROR   
00:45:14    jtreg:test/hotspot/jtreg:vmTestbase_nsk_sysdict      20    20     0     0   
00:45:14 ==============================
00:45:14 TEST SUCCESS
00:45:14 
00:45:15 Finished building target 'run-test' in configuration 'linux-x86_64-server-fastdebug'

real	45m14.839s
user	149m49.850s
sys	13m25.849s

After:

$ time CONF=linux-x86_64-server-fastdebug make run-test TEST=vmTestbase_nsk_sysdict | ts -s
...
00:03:25 ==============================
00:03:25 Test summary
00:03:25 ==============================
00:03:25    TEST                                              TOTAL  PASS  FAIL ERROR   
00:03:25    jtreg:test/hotspot/jtreg:vmTestbase_nsk_sysdict      20    20     0     0   
00:03:25 ==============================
00:03:25 TEST SUCCESS
00:03:25 
00:03:25 Finished building target 'run-test' in configuration 'linux-x86_64-server-fastdebug'

real	3m24.702s
user	119m7.488s
sys	8m11.716s

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8273433: Enable parallelism in vmTestbase_nsk_sysdict tests

Reviewers

• Mikhailo Seledtsov (@mseledts - Committer)
• David Holmes (@dholmes-ora - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5389/head:pull/5389
$ git checkout pull/5389

Update a local copy of the PR:
$ git checkout pull/5389
$ git pull https://git.openjdk.java.net/jdk pull/5389/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5389

View PR using the GUI difftool:
$ git pr show -t 5389

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5389.diff

[bridgekeeper]

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@shipilev The following label will be automatically applied to this pull request:

• hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (3559e1c4)

[mlbridge]

Mailing list message from David Holmes on hotspot-runtime-dev:

Hi Aleksey,

On 7/09/2021 11:24 pm, Aleksey Shipilev wrote:

Current `vmTestbase_nsk_sysdict` suite contains about 20 tests, each running exclusively. There seem to be no reason to run them exclusively, though: they complete in reasonable time, are single-threaded, and consume the usual amount of memory.

I (or someone) will to need to check the history here. There's always a
reason tests were set to run exclusively.

Thanks,
David

[shipilev]

I (or someone) will to need to check the history here. There's always a reason tests were set to run exclusively.

Please do. TEST.properties have no history before OpenJDK, and I found no relevant problems in JBS. These exclusiveAccess.dir-s look as something added for extra safety, but we pay dearly for that extra with test time.

[dholmes-ora]

I suspect it was simply the fact that these were stress tests that resulted in them being executed in isolation. Otherwise if the concurrency level is just wrong I would expect these to cause intermittent failures when run in parallel. I will ask Misha to respond here (can't locate his OpenJDK tag :( ).

[dholmes-ora]

I'm also concerned that hard-wiring this sort of thing is never going to make everyone happy. Probably what we need is a way to disable "exclusiveDirs" processing on the jtreg command-line ... though that would need some finer-level of control to make it useful.

[shipilev]

I suspect it was simply the fact that these were stress tests that resulted in them being executed in isolation. Otherwise if the concurrency level is just wrong I would expect these to cause intermittent failures when run in parallel.

Yeah, I generally agree with this. Blind parallelism improvements should not be done, and we should only allow the stress tests that are not heavy on resources to run in parallel. The tests affected in this PR seem to be rather short, effectively single-threaded, and memory-dense tests, so they seem to qualify for this kind of relaxation. I'll wait for Misha to respond.

[mseledts]

Thanks Alexey. I will run some tests, look into history of tests and will get back to you with my findings tonight (PST) or tomorrow.

[mseledts]

The internal history of the change that restricted parallelism for this test set indicates that it was a wide reaching change covering a wide set of stress tests. Based on Aleksey's comment in the PR the tests "seem to be rather short, effectively single-threaded, and memory-dense tests, so they seem to qualify for this kind of relaxation." I tend to agree with his assessment.

I have also executed these tests with the change on 5 different platforms, 100 times on each platform, and found no issues. The change looks good to me.

[openjdk]

@shipilev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8273433: Enable parallelism in vmTestbase_nsk_sysdict tests

Reviewed-by: mseledtsov, dholmes

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 38 new commits pushed to the master branch:

• 5df2648: 8273218: G1: Rename g1EvacuationInfo to g1EvacInfo
• 12f0b77: 8273516: ProblemList compiler/c2/Test7179138_1.java in -Xcomp mode on win-X64
• 7fd6b0b: 8273242: (test) Refactor to use TestNG for RuntimeTests ExecCommand tests
• a5e4def: 8265489: Stress test times out because of long ObjectSynchronizer::monitors_iterate(...) operation
• 9b5991e: 8273451: Remove unreachable return in mutexLocker::wait
• e680503: 8273185: Rename the term "atomic" in ReferenceProcessor
• ba31eee: 8273109: runtime/cds/appcds/loaderConstraints/DynamicLoaderConstraintsTest times out
• 4d5e1ed: 8272375: Improve phrasing of synthesized descriptions in JavaFX docs
• 267c61a: 8273361: InfoOptsTest is failing in tier1
• f7e9f56: 8273359: CI: ciInstanceKlass::get_canonical_holder() doesn't respect instance size
• ... and 28 more: https://git.openjdk.java.net/jdk/compare/3cd95a2932ab6a2cb39322354f7ad181de8bfc9e...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[shipilev]

OK, good, thanks for testing, @mseledts! I'll be on lookout if these tests start to misbehave after this relaxation.

/integrate

[openjdk]

Going to push as commit 5ca26cb.
Since your change was applied there have been 46 commits pushed to the master branch:

• f6cc173: 8273539: [PPC64] gtest build error after JDK-8264207
• 9690df7: 8273476: G1: refine G1CollectedHeap::par_iterate_regions_array_part_from
• 00e059d: 8133686: HttpURLConnection.getHeaderFields and URLConnection.getRequestProperties methods return field values in reverse order
• aa93111: 8273483: Zero: Clear pending JNI exception check in native method handler
• 8c16f48: 8273487: Zero: Handle "zero" variant in runtime tests
• dc33bd8: 8273315: Parallelize and increase timeouts for java/foreign/TestMatrix.java test
• 5b1dfe4: 8273439: Fix G1CollectedHeap includes and forward declarations
• 6eba443: 8273387: remove some unreferenced gtk-related functions
• 5df2648: 8273218: G1: Rename g1EvacuationInfo to g1EvacInfo
• 12f0b77: 8273516: ProblemList compiler/c2/Test7179138_1.java in -Xcomp mode on win-X64
• ... and 36 more: https://git.openjdk.java.net/jdk/compare/3cd95a2932ab6a2cb39322354f7ad181de8bfc9e...master

Your commit was automatically rebased without conflicts.

[openjdk]

@shipilev Pushed as commit 5ca26cb.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.