Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8247980: Exclusive execution of java/util/stream tests slows down tier1 #5189

Conversation

shipilev
Copy link
Contributor

@shipilev shipilev commented Aug 19, 2021

See the bug report for more details. I would appreciate if people with their corporate testing systems would run this through their systems to avoid surprises. (ping @RealCLanger, @iignatev).


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8247980: Exclusive execution of java/util/stream tests slows down tier1

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5189/head:pull/5189
$ git checkout pull/5189

Update a local copy of the PR:
$ git checkout pull/5189
$ git pull https://git.openjdk.java.net/jdk pull/5189/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5189

View PR using the GUI difftool:
$ git pr show -t 5189

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5189.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Aug 19, 2021

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Aug 19, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 19, 2021

@shipilev The following label will be automatically applied to this pull request:

  • core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the core-libs label Aug 19, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Aug 19, 2021

Webrevs

@iignatev
Copy link
Member

@iignatev iignatev commented Aug 19, 2021

Hi @shipilev ,

I've submitted testing for this patch in our system. meanwhile, I'd like to hear @PaulSandoz's and/or @AlanBateman's option on this as I don't think that the underlying issue, that forced us to add these tests to exclusiveAccess.dirs by 8058204, was resolved.

Thanks,
-- Igor

@PaulSandoz
Copy link
Member

@PaulSandoz PaulSandoz commented Aug 19, 2021

Struggling to recall all the details. I think in part the tests were made exclusive to avoid the greedy parallel tests potentially competing with the execution of other greedy parallel stream tests and other independent tests.

The issue may be petrubed when using machines with a large number of cores, as cost of greedly using all hardware threads is counter productive to the data sizes. It may be useful if we could limit the parallelism of stream tests on machines with large cores. I don't recall if we currently do that or have explored that.

There may be no ideal configuration that fits machines with small and large cores. If there is a prioritization i would lean towards optimization by default for the automated test infra with options to modify that default for others running the tests.

@RealCLanger
Copy link
Contributor

@RealCLanger RealCLanger commented Aug 24, 2021

I have added this to our internal testing, let's see how it goes. Notifying @ArnoZeller about this.

@iignatev
Copy link
Member

@iignatev iignatev commented Aug 25, 2021

the testing in our infra returned green.

@mlbridge
Copy link

@mlbridge mlbridge bot commented Sep 1, 2021

Mailing list message from Alan Bateman on core-libs-dev:

On 19/08/2021 17:10, Igor Ignatyev wrote:

On Thu, 19 Aug 2021 15:15:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

See the bug report for more details. I would appreciate if people with their corporate testing systems would run this through their systems to avoid surprises. (ping @RealCLanger, @iignatev).
Hi @shipilev ,

I've submitted testing for this patch in our system. meanwhile, I'd like to hear @PaulSandoz's and/or @AlanBateman's option on this as I don't think that the underlying issue, that forced us to add these tests to `exclusiveAccess.dirs` by [8058204](https://bugs.openjdk.java.net/browse/JDK-8058204), was resolved.

These tests use all cores so were historically problematic with several
agents VMs running concurrently, lots of timeouts. I don't recall if the
reason for putting then on exclusiveAccess.dirs was release or debug
builds. Getting fresh results would be useful.

-Alan

@shipilev
Copy link
Contributor Author

@shipilev shipilev commented Sep 2, 2021

I have added this to our internal testing, let's see how it goes. Notifying @ArnoZeller about this.

Any good or bad results?

@shipilev
Copy link
Contributor Author

@shipilev shipilev commented Sep 13, 2021

@ArnoZeller, have your runs identify any problems with these tests?

I wonder if we want to integrate this PR, and see if we see any problems going forward. If we do, and those problems look environmental or intrinsic to the tests, we can revert the patch. Thoughts?

@ArnoZeller
Copy link
Contributor

@ArnoZeller ArnoZeller commented Sep 15, 2021

I haven't recognized problems I can directly match to this change.
But I should also state, that we parallelize not by using several agents in one jtreg tests - we use no agents and do not set concurrency. Currently we split the jtreg test in groups and each group is executed by a jtreg runner with concurrency one. But the runner do run in parallel if the machine is big enough.
With this setup I do not expect influence by your change because we still execute the test in serial order - even after this change.
At least that is how I understand the effect of exclusiveAccess.dirs .

@shipilev
Copy link
Contributor Author

@shipilev shipilev commented Sep 16, 2021

All right, thank you @ArnoZeller!

@iignatev, @PaulSandoz, @AlanBateman -- are you good with integrating this?

@openjdk
Copy link

@openjdk openjdk bot commented Sep 16, 2021

@shipilev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8247980: Exclusive execution of java/util/stream tests slows down tier1

Reviewed-by: iignatyev

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 310 new commits pushed to the master branch:

  • 59b2478: 8273659: Replay compilation crashes with SIGSEGV since 8271911
  • 5e4d09c: 8273300: Check Mutex ranking during a safepoint
  • c86e24d: 8271954: C2: assert(false) failed: Bad graph detected in build_loop_late
  • 14dc517: 8273372: Remove scavenge trace message in psPromotionManager
  • 241ac89: 8273606: Zero: SPARC64 build fails with si_band type mismatch
  • 181292d: 8273801: Handle VMTYPE for "core" VM variant
  • 09ecb11: 8273806: compiler/cpuflags/TestSSE4Disabled.java should test for CPU feature explicitly
  • 99cfc16: 8273805: gc/g1/TestGCLogMessages.java test should handle non-JFR configs
  • 1c5de8b: 8273807: Zero: Drop incorrect test block from compiler/startup/NumCompilerThreadsCheck.java
  • 46af82e: 8273804: Platform.isTieredSupported should handle the no-compiler case
  • ... and 300 more: https://git.openjdk.java.net/jdk/compare/79a06df8113ba1da55db5c38fe34608c3507c223...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Sep 16, 2021
@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Sep 17, 2021

@iignatev, @PaulSandoz, @AlanBateman -- are you good with integrating this?

No objection from me. If issues arise then I assume it can be discussed again and we can figure out something that works for whatever CI or environment where it is problematic.

@shipilev
Copy link
Contributor Author

@shipilev shipilev commented Sep 17, 2021

No objection from me. If issues arise then I assume it can be discussed again and we can figure out something that works for whatever CI or environment where it is problematic.

Yes, I think if tests start to fail, we look into concrete cases. If we see those failures are hard to fix, we revert this patch, and wait a few years again :)

@PaulSandoz
Copy link
Member

@PaulSandoz PaulSandoz commented Sep 17, 2021

@iignatev, @PaulSandoz, @AlanBateman -- are you good with integrating this?

Yes, lets give it a try.

@shipilev
Copy link
Contributor Author

@shipilev shipilev commented Sep 20, 2021

Ok, let's try!

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Sep 20, 2021

Going to push as commit 544193a.
Since your change was applied there have been 334 commits pushed to the master branch:

  • 1f8af52: 8273314: Add tier4 test groups
  • 699865f: 8259039: Passing different version to --release flag than javac version output warning
  • f71df14: 8273187: jtools tests fail with missing markerName check
  • 6f3e40c: 8273825: TestIRMatching.java fails after JDK-8266550
  • a561eac: 8273895: compiler/ciReplay/TestVMNoCompLevel.java fails due to wrong data size with TieredStopAtLevel=2,3
  • d2388b7: 8273959: Some metaspace diagnostic switches should be develop
  • dc7f452: 8273815: move have_special_privileges to os_posix for POSIX platforms
  • 7c9868c: 8273454: C2: Transform (-a)(-b) into ab
  • bb9d142: 8273958: gtest/MetaspaceGtests executes unnecessary tests in debug builds
  • 2a2e919: 8273685: Remove jtreg tag manual=yesno for java/awt/Graphics/LCDTextAndGraphicsState.java & show test instruction
  • ... and 324 more: https://git.openjdk.java.net/jdk/compare/79a06df8113ba1da55db5c38fe34608c3507c223...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Sep 20, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Sep 20, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Sep 20, 2021

@shipilev Pushed as commit 544193a.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@shipilev shipilev deleted the JDK-8247980-non-exclusive-stream-tests branch Oct 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs integrated
6 participants