Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8273433: Enable parallelism in vmTestbase_nsk_sysdict tests #5389

Conversation

shipilev
Copy link
Member

@shipilev shipilev commented Sep 7, 2021

Current vmTestbase_nsk_sysdict suite contains about 20 tests, each running exclusively. There seem to be no reason to run them exclusively, though: they complete in reasonable time, are single-threaded, and consume the usual amount of memory.

We should consider enabling parallelism for them and get improved test performance. Currently it is blocked by TEST.properties with exclusiveAccess.dirs directives in them.

Motivational performance improvements below.

Before:

$ time CONF=linux-x86_64-server-fastdebug make run-test TEST=vmTestbase_nsk_sysdict | ts -s
...
00:45:14 ==============================
00:45:14 Test summary
00:45:14 ==============================
00:45:14    TEST                                              TOTAL  PASS  FAIL ERROR   
00:45:14    jtreg:test/hotspot/jtreg:vmTestbase_nsk_sysdict      20    20     0     0   
00:45:14 ==============================
00:45:14 TEST SUCCESS
00:45:14 
00:45:15 Finished building target 'run-test' in configuration 'linux-x86_64-server-fastdebug'

real	45m14.839s
user	149m49.850s
sys	13m25.849s

After:

$ time CONF=linux-x86_64-server-fastdebug make run-test TEST=vmTestbase_nsk_sysdict | ts -s
...
00:03:25 ==============================
00:03:25 Test summary
00:03:25 ==============================
00:03:25    TEST                                              TOTAL  PASS  FAIL ERROR   
00:03:25    jtreg:test/hotspot/jtreg:vmTestbase_nsk_sysdict      20    20     0     0   
00:03:25 ==============================
00:03:25 TEST SUCCESS
00:03:25 
00:03:25 Finished building target 'run-test' in configuration 'linux-x86_64-server-fastdebug'

real	3m24.702s
user	119m7.488s
sys	8m11.716s

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8273433: Enable parallelism in vmTestbase_nsk_sysdict tests

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5389/head:pull/5389
$ git checkout pull/5389

Update a local copy of the PR:
$ git checkout pull/5389
$ git pull https://git.openjdk.java.net/jdk pull/5389/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5389

View PR using the GUI difftool:
$ git pr show -t 5389

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5389.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Sep 7, 2021

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Sep 7, 2021
@openjdk
Copy link

openjdk bot commented Sep 7, 2021

@shipilev The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Sep 7, 2021
@mlbridge
Copy link

mlbridge bot commented Sep 7, 2021

Webrevs

@mlbridge
Copy link

mlbridge bot commented Sep 7, 2021

Mailing list message from David Holmes on hotspot-runtime-dev:

Hi Aleksey,

On 7/09/2021 11:24 pm, Aleksey Shipilev wrote:

Current `vmTestbase_nsk_sysdict` suite contains about 20 tests, each running exclusively. There seem to be no reason to run them exclusively, though: they complete in reasonable time, are single-threaded, and consume the usual amount of memory.

I (or someone) will to need to check the history here. There's always a
reason tests were set to run exclusively.

Thanks,
David

@shipilev
Copy link
Member Author

shipilev commented Sep 7, 2021

I (or someone) will to need to check the history here. There's always a reason tests were set to run exclusively.

Please do. TEST.properties have no history before OpenJDK, and I found no relevant problems in JBS. These exclusiveAccess.dir-s look as something added for extra safety, but we pay dearly for that extra with test time.

@dholmes-ora
Copy link
Member

I suspect it was simply the fact that these were stress tests that resulted in them being executed in isolation. Otherwise if the concurrency level is just wrong I would expect these to cause intermittent failures when run in parallel. I will ask Misha to respond here (can't locate his OpenJDK tag :( ).

@dholmes-ora
Copy link
Member

I'm also concerned that hard-wiring this sort of thing is never going to make everyone happy. Probably what we need is a way to disable "exclusiveDirs" processing on the jtreg command-line ... though that would need some finer-level of control to make it useful.

@shipilev
Copy link
Member Author

shipilev commented Sep 8, 2021

I suspect it was simply the fact that these were stress tests that resulted in them being executed in isolation. Otherwise if the concurrency level is just wrong I would expect these to cause intermittent failures when run in parallel.

Yeah, I generally agree with this. Blind parallelism improvements should not be done, and we should only allow the stress tests that are not heavy on resources to run in parallel. The tests affected in this PR seem to be rather short, effectively single-threaded, and memory-dense tests, so they seem to qualify for this kind of relaxation. I'll wait for Misha to respond.

@mseledts
Copy link
Member

mseledts commented Sep 8, 2021

Thanks Alexey. I will run some tests, look into history of tests and will get back to you with my findings tonight (PST) or tomorrow.

Copy link
Member

@mseledts mseledts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The internal history of the change that restricted parallelism for this test set indicates that it was a wide reaching change covering a wide set of stress tests. Based on Aleksey's comment in the PR the tests "seem to be rather short, effectively single-threaded, and memory-dense tests, so they seem to qualify for this kind of relaxation." I tend to agree with his assessment.

I have also executed these tests with the change on 5 different platforms, 100 times on each platform, and found no issues. The change looks good to me.

@openjdk
Copy link

openjdk bot commented Sep 9, 2021

@shipilev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8273433: Enable parallelism in vmTestbase_nsk_sysdict tests

Reviewed-by: mseledtsov, dholmes

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 38 new commits pushed to the master branch:

  • 5df2648: 8273218: G1: Rename g1EvacuationInfo to g1EvacInfo
  • 12f0b77: 8273516: ProblemList compiler/c2/Test7179138_1.java in -Xcomp mode on win-X64
  • 7fd6b0b: 8273242: (test) Refactor to use TestNG for RuntimeTests ExecCommand tests
  • a5e4def: 8265489: Stress test times out because of long ObjectSynchronizer::monitors_iterate(...) operation
  • 9b5991e: 8273451: Remove unreachable return in mutexLocker::wait
  • e680503: 8273185: Rename the term "atomic" in ReferenceProcessor
  • ba31eee: 8273109: runtime/cds/appcds/loaderConstraints/DynamicLoaderConstraintsTest times out
  • 4d5e1ed: 8272375: Improve phrasing of synthesized descriptions in JavaFX docs
  • 267c61a: 8273361: InfoOptsTest is failing in tier1
  • f7e9f56: 8273359: CI: ciInstanceKlass::get_canonical_holder() doesn't respect instance size
  • ... and 28 more: https://git.openjdk.java.net/jdk/compare/3cd95a2932ab6a2cb39322354f7ad181de8bfc9e...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Sep 9, 2021
@shipilev
Copy link
Member Author

shipilev commented Sep 9, 2021

OK, good, thanks for testing, @mseledts! I'll be on lookout if these tests start to misbehave after this relaxation.

/integrate

@openjdk
Copy link

openjdk bot commented Sep 9, 2021

Going to push as commit 5ca26cb.
Since your change was applied there have been 46 commits pushed to the master branch:

  • f6cc173: 8273539: [PPC64] gtest build error after JDK-8264207
  • 9690df7: 8273476: G1: refine G1CollectedHeap::par_iterate_regions_array_part_from
  • 00e059d: 8133686: HttpURLConnection.getHeaderFields and URLConnection.getRequestProperties methods return field values in reverse order
  • aa93111: 8273483: Zero: Clear pending JNI exception check in native method handler
  • 8c16f48: 8273487: Zero: Handle "zero" variant in runtime tests
  • dc33bd8: 8273315: Parallelize and increase timeouts for java/foreign/TestMatrix.java test
  • 5b1dfe4: 8273439: Fix G1CollectedHeap includes and forward declarations
  • 6eba443: 8273387: remove some unreferenced gtk-related functions
  • 5df2648: 8273218: G1: Rename g1EvacuationInfo to g1EvacInfo
  • 12f0b77: 8273516: ProblemList compiler/c2/Test7179138_1.java in -Xcomp mode on win-X64
  • ... and 36 more: https://git.openjdk.java.net/jdk/compare/3cd95a2932ab6a2cb39322354f7ad181de8bfc9e...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Sep 9, 2021
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 9, 2021
@openjdk
Copy link

openjdk bot commented Sep 9, 2021

@shipilev Pushed as commit 5ca26cb.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@shipilev shipilev deleted the JDK-8273433-vmTestbase-nsk-sysdict-par branch September 13, 2021 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime hotspot-runtime-dev@openjdk.org integrated Pull request has been integrated
3 participants