Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8315936: Parallelize gc/stress/TestStressG1Humongous.java test #15943

Closed
wants to merge 3 commits into from

Conversation

ianrichr
Copy link
Contributor

@ianrichr ianrichr commented Sep 27, 2023

TestStressG1Humongous runs in hotspot:tier4 and takes about 900 seconds to run. It is one of the slowest tests in tier4, and it runs when concurrency is generally low due to exclusive adjacent tests. Thus, it limits the effective parallelism of tier4.

In test main method, there are a few independent test cases, which can be expressed in jtreg to gain parallel execution.

TestStressG1Humongous current runtime: 4917.51s user 410.97s system 549% cpu 16:10.07 total
TestStressG1Humongous parallelized runtime: 4672.49s user 510.17s system 849% cpu 10:10.05 total

This change takes the previous run() methods out of the main method and instead passes the parameters as part of the @run jtreg test annotation, allowing them to be run in parallel.

The last test runs the longest on its own, and this is why the runtime can't be reduced under the 10 minute mark.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

  • JDK-8315936: Parallelize gc/stress/TestStressG1Humongous.java test (Enhancement - P4)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15943/head:pull/15943
$ git checkout pull/15943

Update a local copy of the PR:
$ git checkout pull/15943
$ git pull https://git.openjdk.org/jdk.git pull/15943/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15943

View PR using the GUI difftool:
$ git pr show -t 15943

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15943.diff

Webrev

Link to Webrev Comment

@ianrichr ianrichr changed the title Parallelize gc/stress/TestStressG1Humongous.java test 8315936: Parallelize gc/stress/TestStressG1Humongous.java test Sep 27, 2023
@bridgekeeper
Copy link

bridgekeeper bot commented Sep 27, 2023

👋 Welcome back ianrichr! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Sep 27, 2023

@ianrichr The following label will be automatically applied to this pull request:

  • hotspot-gc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label Sep 27, 2023
Copy link
Member

@shipilev shipilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay, but has a few nits to address.

It is common to check the arg length first:

public static void main(String[] args) throws Exception {
    if (args.length != 4) {
        throw new IllegalArgumentException("Test expects 4 arguments");
    }
}

double humongousSize = Double.parseDouble(args[2]);
int timeout = Integer.parseInt(args[3]);

run(heapSize, regionSize, threads, humongousSize, timeout);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can remove the run() and inline the whole thing into main(). Just remove the lines in the middle.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made these changes. Thanks @shipilev

@openjdk openjdk bot added the rfr Pull request is ready for review label Sep 27, 2023
@mlbridge
Copy link

mlbridge bot commented Sep 27, 2023

Webrevs

Copy link
Member

@shipilev shipilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All right, I think only one nit is left. Maybe @tschatzl would like to take a look as well.

/reviewers 2

@@ -23,15 +23,42 @@

package gc.stress;

/*
* @test TestStressG1Humongous
/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and later, this is misaligned. All stars should align, like before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stars ⭐ are aligned now, thanks!

@openjdk
Copy link

openjdk bot commented Sep 27, 2023

@ianrichr This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315936: Parallelize gc/stress/TestStressG1Humongous.java test

Reviewed-by: shade, tschatzl

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 26 new commits pushed to the master branch:

  • fc98998: 8316580: HttpClient with StructuredTaskScope does not close when a task fails
  • 065203d: 8313631: SA: stack trace printed using "where" command does not show class name
  • 0c55887: 8309599: WeakHandle and OopHandle release should clear obj pointer
  • 1230aed: 8316885: jcmd: Compiler.CodeHeap_Analytics cmd does not inform about missing aggregate
  • d0c1444: 8316957: Serial: Change GenCollectedHeap to SerialHeap inside gc/serial folder
  • bc5a39b: 8317144: Exclude sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java on Linux ppc64le
  • 3481ecb: 8316159: Update BoxLayout sample image for crisper edges
  • 7981251: 8316970: Add internal annotation to mark restricted methods
  • 77fac0f: 8316906: Clarify TLABWasteTargetPercent flag
  • 2d154fc: 8316566: RISC-V: Zero extended narrow oop passed to Atomic::cmpxchg
  • ... and 16 more: https://git.openjdk.org/jdk/compare/fee9d3362c76a046bb5160b90536545e7e9a5ce9...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@shipilev, @tschatzl) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Sep 27, 2023
@openjdk
Copy link

openjdk bot commented Sep 27, 2023

@shipilev
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

@openjdk openjdk bot removed the ready Pull request is ready to be integrated label Sep 27, 2023
@tschatzl
Copy link
Contributor

I can take this change for a spin internally - the main reason why this test is run separately is that it tends to make other similar tests fail (time out) because the test needs so much cpu time (particularly without the "new" remset).

In the worst case, if this causes timeouts, we would probably need to revert this change.

@tschatzl
Copy link
Contributor

Particularly smaller systems (macs?) may be affected.

@ianrichr
Copy link
Contributor Author

Thanks @tschatzl I'll be curious to hear the results.

Copy link
Contributor

@tschatzl tschatzl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good at first glance, but I'll monitor our CI for issues

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Sep 28, 2023
@ianrichr
Copy link
Contributor Author

/integrate

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Sep 28, 2023
@openjdk
Copy link

openjdk bot commented Sep 28, 2023

@ianrichr
Your change (at version ba9820d) is now ready to be sponsored by a Committer.

@shipilev
Copy link
Member

/sponsor

@openjdk
Copy link

openjdk bot commented Sep 28, 2023

Going to push as commit 3f19df6.
Since your change was applied there have been 26 commits pushed to the master branch:

  • fc98998: 8316580: HttpClient with StructuredTaskScope does not close when a task fails
  • 065203d: 8313631: SA: stack trace printed using "where" command does not show class name
  • 0c55887: 8309599: WeakHandle and OopHandle release should clear obj pointer
  • 1230aed: 8316885: jcmd: Compiler.CodeHeap_Analytics cmd does not inform about missing aggregate
  • d0c1444: 8316957: Serial: Change GenCollectedHeap to SerialHeap inside gc/serial folder
  • bc5a39b: 8317144: Exclude sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java on Linux ppc64le
  • 3481ecb: 8316159: Update BoxLayout sample image for crisper edges
  • 7981251: 8316970: Add internal annotation to mark restricted methods
  • 77fac0f: 8316906: Clarify TLABWasteTargetPercent flag
  • 2d154fc: 8316566: RISC-V: Zero extended narrow oop passed to Atomic::cmpxchg
  • ... and 16 more: https://git.openjdk.org/jdk/compare/fee9d3362c76a046bb5160b90536545e7e9a5ce9...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Sep 28, 2023
@openjdk openjdk bot closed this Sep 28, 2023
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Sep 28, 2023
@openjdk
Copy link

openjdk bot commented Sep 28, 2023

@shipilev @ianrichr Pushed as commit 3f19df6.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@ianrichr
Copy link
Contributor Author

/backport jdk21u

@openjdk
Copy link

openjdk bot commented Nov 21, 2023

@ianrichr the backport was successfully created on the branch ianrichr-backport-3f19df68 in my personal fork of openjdk/jdk21u. To create a pull request with this backport targeting openjdk/jdk21u:master, just click the following link:

➡️ Create pull request

The title of the pull request is automatically filled in correctly and below you find a suggestion for the pull request body:

Hi all,

This pull request contains a backport of commit 3f19df68 from the openjdk/jdk repository.

The commit being backported was authored by Ian Myers on 28 Sep 2023 and was reviewed by Aleksey Shipilev and Thomas Schatzl.

Thanks!

If you need to update the source branch of the pull then run the following commands in a local clone of your personal fork of openjdk/jdk21u:

$ git fetch https://github.com/openjdk-bots/jdk21u.git ianrichr-backport-3f19df68:ianrichr-backport-3f19df68
$ git checkout ianrichr-backport-3f19df68
# make changes
$ git add paths/to/changed/files
$ git commit --message 'Describe additional changes made'
$ git push https://github.com/openjdk-bots/jdk21u.git ianrichr-backport-3f19df68

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-gc hotspot-gc-dev@openjdk.org integrated Pull request has been integrated
3 participants