8254699: Suboptimal PreTouchParallelChunkSize defaults and limits #1503

amitdpawar · 2020-11-29T16:47:58Z

This PR fixes lower and default value of JVM flag PreTouchParallelChunkSize. Its default value is 1GB and is used by both G1GC and ParallelGC to pretouch the pages. Following test showed that reducing the chunk size improves JVM startup time and GC pause time.

Tests are: (Test machine 2P 64C/128T with 1TB memory)

JVM startup time test with AdaptiveSizePolicy disabled: Pretouch 1TB of memory with/without transparent large page support and used time command to measure the time taken.
Command: time ./jdk/bin/java -XX:+AlwaysPreTouch -XX:+-Xmx900g -Xms900g -Xmn800g -XX:SurvivorRatio=400 -Xlog:gc*=debug:file=gc.log -XX:ParallelGCThreads=128 -XX:PreTouchParallelChunkSize= -version
JVM startup and GC pause time test with AdaptiveSizePolicy enabled: SPECjbb composite run with 1TB heap and transparent large page support was enabled.

Test results are recorded in XL file. PreTouchParallelChunkSize_TestResults.xlsx

Test results shows:

With AdaptiveSizePolicy disabled.
1. G1GC improved upto ~14% on large page disabled and ~5% on enabled.
2. ParallelGC improved upto ~15% on large page disabled and ~5% on enabled.
3. Tests showed improvement from 64KB for default page size and 2MB for lage page size.
4. Please check "JVM_Startup_Summary" sheet in XL file for more detail.
SPECjbb composite test with UseAdaptiveSizePolicy + UseLargePages enabled.
1. Pretouch takes up-to 30-90% less time for memory range 32MB-4GB. This happens because memory less than 1GB also pretouched with multiple threads.
2. Same also helps to bring down GC pause time and this is dependent on memory size. Effect is larger when expansion size is smaller.
3. Please check SPECjbb_Summary sheet in XL file for more detail.

Default value of PreTouchParallelChunkSize is changed to 4MB and based your suggestion it can be changed to right value. Please check and review this PR.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8254699: Suboptimal PreTouchParallelChunkSize defaults and limits

Reviewers

Thomas Schatzl (@tschatzl - Reviewer) ⚠️ Review applies to ef6fa41
Stefan Johansson (@kstefanj - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/1503/head:pull/1503
$ git checkout pull/1503

…om 1g to 4m.

bridgekeeper · 2020-11-29T16:48:48Z

👋 Welcome back amitdpawar! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-11-29T16:49:46Z

@amitdpawar The following label will be automatically applied to this pull request:

hotspot-gc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2020-11-29T16:53:00Z

Webrevs

tschatzl

Looks good, but could you undo the changes in pretouchTask.cpp? These break the rule to have all gang tasks with a "Running .... with ... workers" message. Also, this message is then printed for all pretouch actions - even when resizing the heap which can be quite annoying.

Instead, the method could use a GCTraceTime instance to time the method. However I do not think this is really necessary or desired - imho in this case the caller should decide on whether it wants some log output, but others may have a different opinion :)

Since a CSR is needed for changes to product flags like this, I started one with JDK-8257419. Probably also needs a release note.

/csr needed

amitdpawar · 2020-12-01T11:13:18Z

Thanks Thomas for your reply. Log message was changed to include the time to make it easier for testing and reviewing. If not required will revert it back. Please suggest.

Thanks,
Amit

tschatzl · 2020-12-01T11:40:03Z

Please remove this what looks like debug code.

kstefanj · 2020-12-01T11:42:13Z

Thanks Thomas for your reply. Log message was changed to include the time to make it easier for testing and reviewing. If not required will revert it back. Please suggest.

I agree with Thomas, I think we should revert the changes done in pretouchTask.cpp.

amitdpawar · 2020-12-01T16:38:51Z

Done.

Thanks,
Amit

tschatzl

Lgtm.

We need to wait until the CSR has been approved. This typically happens on Thursdays.

openjdk · 2020-12-01T16:42:51Z

@amitdpawar This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254699: Suboptimal PreTouchParallelChunkSize defaults and limits

Reviewed-by: tschatzl, sjohanss

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 122 new commits pushed to the master branch:

66a2e70: 8255845: Memory leak in imageFile.cpp
fa58671: 8257020: [JVMCI] enable a JVMCICompiler to specify which GCs it supports
129c377: 8257594: C2 compiled checkcast of non-null object triggers endless deoptimization/recompilation cycle
e4497c9: 8256718: Obsolete the long term deprecated and aliased Trace flags
4a267f1: 8244847: Linux/PPC: runtime/CompressedOops/CompressedClassPointers: smallHeapTest fails
b44a329: 8256864: [windows] Improve tracing for mapping errors
ae1eb28: 8257604: JNI_ArgumentPusherVaArg leaks valist
4169d96: 8257143: Enable JVMCI code installation tests on AArch64
a5a034b: 8257617: TestLinkPlatform fails with new Java source version
d80ae05: 8166596: TLS support for the EdDSA signature algorithm
... and 112 more: https://git.openjdk.java.net/jdk/compare/b1d149935603a4b1cc2acbaf0de01e42622d4419...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@tschatzl, @kstefanj) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

tschatzl · 2020-12-01T16:44:04Z

/csr

openjdk · 2020-12-01T16:44:28Z

@tschatzl this pull request will not be integrated until the CSR request JDK-8257419 for issue JDK-8254699 has been approved.

amitdpawar · 2020-12-01T16:51:20Z

Thanks Thomas and Stefan for reviewing and approving the changes. Will wait until csr approval.

kstefanj · 2020-12-02T10:04:32Z

I did some performance runs and found that on Windows this change will not speed up pre-touching. I see some quite big regressions in some cases. So I don't think we can do this change for all platforms without doing more benchmarking.

But since it looks good on Linux, one solution would be to make PreTouchParallelChunkSize a platform-dependent flag and set it to 4M for Linux and keep it at 1G for the others until we can do more investigations.

For guidance on how to make it a platform-dependent flag you can look at how UseLargePages is handled.

amitdpawar · 2020-12-02T14:11:57Z

I was doubtful about the improvement regarding other platforms and thanks for testing and verifying. I will make it platform-specific as per your suggestion.

On other platform, this improvement is not seen for smaller or lesser memory range also right ? similar too SPECJbb_Summary sheet in Excel file.

kstefanj · 2020-12-02T14:36:08Z

I have not done extensive measurements, I basically ran some startup benchmarks with:

-XX:+AlwaysPreTouch -Xms8g -Xmx8g
-XX:+AlwaysPreTouch -Xms8g -Xmx8g -XX:PreTouchParallelChunkSize=4m
-XX:+AlwaysPreTouch -Xms8g -Xmx8g -XX:PreTouchParallelChunkSize=128m

And on Windows going with the current default is the clear winner, while on Linux using 4M gives best results. Given that and your tests I think it is fairly safe to use 4M for Linux, but for the other OSes we need to do more measurements before changing to a different value.

amitdpawar · 2020-12-02T14:52:04Z

OK and I will make it platform specific as suggested.

Thanks,
Amit

amitdpawar · 2020-12-03T11:37:01Z

PreTouchParallelChunkSize is changed to platform-dependent as suggested. Please check now.

To make testing easier created two scripts and attached as zip file run_pretouch_test.zip.

Script "run_pretouch_test.sh" will run pretouch test by configuring "from space" to 1MB to 8GB for chunk sizes 64KB to 1GB. Tested only on Linux for both default and large pages.
Script "print_timetaken.sh" will collect and print time taken from all the generated GC log files. Time taken will be extracted for only "from space" and it will print table similar to shown in Excel file. PreTouchWithDifferentMemorySize.xlsx
Should have build using git id "8ef5ed7c64808c6b039baa0751afa7ff017b318d" else it wont work. This particular revision dumps time for PretouchTask.

Thanks,
Amit

…t value set to 4M from 1G for Linux only.

kstefanj

Looks good.

amitdpawar · 2020-12-03T15:24:00Z

Thanks for approving and will issue Integrate command now.

amitdpawar · 2020-12-03T15:33:03Z

/Integrate

openjdk · 2020-12-03T15:33:33Z

@amitdpawar
Your change (at version 4397e43) is now ready to be sponsored by a Committer.

tschatzl · 2020-12-03T15:34:17Z

/integrate

openjdk · 2020-12-03T15:34:22Z

@tschatzl Only the author (@amitdpawar) is allowed to issue the integrate command. As this PR is ready to be sponsored, and you are an eligible sponsor, did you mean to issue the /sponsor command?

tschatzl · 2020-12-03T19:04:28Z

/sponsor

openjdk · 2020-12-03T19:05:18Z

@tschatzl @amitdpawar Since your change was applied there have been 129 commits pushed to the master branch:

e29ee5b: 8257641: Shenandoah: Query is_at_shenandoah_safepoint() from control thread should return false
55f5542: 8026976: ECParameters, Point does not match field size
d3f3c32: 8255742: PrintInlining as compiler directive doesn't print virtual calls
6c9482e: 8257561: Some code is not vectorized after 8251925 and 8250607
70517c8: 8257642: CipherByteBufferOverwriteTest copyright issue
e836396: 8257436: [aarch64] Regressions in ArrayCopyUnalignedDst.testByte/testChar for 65-78 bytes when UseSIMDForMemoryOps is on
b170c83: 8257591: Remove suppression of record preview related warnings in java.lang
66a2e70: 8255845: Memory leak in imageFile.cpp
fa58671: 8257020: [JVMCI] enable a JVMCICompiler to specify which GCs it supports
129c377: 8257594: C2 compiled checkcast of non-null object triggers endless deoptimization/recompilation cycle
... and 119 more: https://git.openjdk.java.net/jdk/compare/b1d149935603a4b1cc2acbaf0de01e42622d4419...master

Your commit was automatically rebased without conflicts.

Pushed as commit 805d058.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Fix lower limit and change PreTouchParallelChunkSize default value fr…

8ef5ed7

…om 1g to 4m.

amitdpawar changed the title ~~Suboptimal PreTouchParallelChunkSize defaults and limits~~ 8254699: Suboptimal PreTouchParallelChunkSize defaults and limits Nov 29, 2020

openjdk bot added hotspot-gc hotspot-gc-dev@openjdk.org rfr Pull request is ready for review labels Nov 29, 2020

tschatzl suggested changes Nov 30, 2020

View reviewed changes

Reverted changes in pretouchTask.cpp.

ef6fa41

tschatzl approved these changes Dec 1, 2020

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Dec 1, 2020

openjdk bot added the csr Pull request needs approved CSR before integration label Dec 1, 2020

openjdk bot removed the ready Pull request is ready to be integrated label Dec 1, 2020

openjdk bot added ready Pull request is ready to be integrated and removed csr Pull request needs approved CSR before integration labels Dec 1, 2020

openjdk bot added hotspot hotspot-dev@openjdk.org and removed hotspot-gc hotspot-gc-dev@openjdk.org labels Dec 3, 2020

PreTouchParallelChunkSize is changed to platform-dependent and defaul…

4397e43

…t value set to 4M from 1G for Linux only.

kstefanj approved these changes Dec 3, 2020

View reviewed changes

openjdk bot added the sponsor Pull request is ready to be sponsored label Dec 3, 2020

openjdk bot closed this Dec 3, 2020

openjdk bot added integrated Pull request has been integrated and removed sponsor Pull request is ready to be sponsored ready Pull request is ready to be integrated rfr Pull request is ready for review labels Dec 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8254699: Suboptimal PreTouchParallelChunkSize defaults and limits #1503

8254699: Suboptimal PreTouchParallelChunkSize defaults and limits #1503

amitdpawar commented Nov 29, 2020 •

edited by openjdk bot

Loading

bridgekeeper bot commented Nov 29, 2020

openjdk bot commented Nov 29, 2020

mlbridge bot commented Nov 29, 2020 •

edited

Loading

tschatzl left a comment

amitdpawar commented Dec 1, 2020

tschatzl commented Dec 1, 2020

kstefanj commented Dec 1, 2020

amitdpawar commented Dec 1, 2020

tschatzl left a comment

openjdk bot commented Dec 1, 2020 •

edited

Loading

tschatzl commented Dec 1, 2020

openjdk bot commented Dec 1, 2020

amitdpawar commented Dec 1, 2020

kstefanj commented Dec 2, 2020

amitdpawar commented Dec 2, 2020

kstefanj commented Dec 2, 2020

amitdpawar commented Dec 2, 2020

amitdpawar commented Dec 3, 2020

kstefanj left a comment

amitdpawar commented Dec 3, 2020

amitdpawar commented Dec 3, 2020

openjdk bot commented Dec 3, 2020

tschatzl commented Dec 3, 2020

openjdk bot commented Dec 3, 2020

tschatzl commented Dec 3, 2020

openjdk bot commented Dec 3, 2020

8254699: Suboptimal PreTouchParallelChunkSize defaults and limits #1503

8254699: Suboptimal PreTouchParallelChunkSize defaults and limits #1503

Conversation

amitdpawar commented Nov 29, 2020 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Download

bridgekeeper bot commented Nov 29, 2020

openjdk bot commented Nov 29, 2020

mlbridge bot commented Nov 29, 2020 • edited Loading

Webrevs

tschatzl left a comment

Choose a reason for hiding this comment

amitdpawar commented Dec 1, 2020

tschatzl commented Dec 1, 2020

kstefanj commented Dec 1, 2020

amitdpawar commented Dec 1, 2020

tschatzl left a comment

Choose a reason for hiding this comment

openjdk bot commented Dec 1, 2020 • edited Loading

tschatzl commented Dec 1, 2020

openjdk bot commented Dec 1, 2020

amitdpawar commented Dec 1, 2020

kstefanj commented Dec 2, 2020

amitdpawar commented Dec 2, 2020

kstefanj commented Dec 2, 2020

amitdpawar commented Dec 2, 2020

amitdpawar commented Dec 3, 2020

kstefanj left a comment

Choose a reason for hiding this comment

amitdpawar commented Dec 3, 2020

amitdpawar commented Dec 3, 2020

openjdk bot commented Dec 3, 2020

tschatzl commented Dec 3, 2020

openjdk bot commented Dec 3, 2020

tschatzl commented Dec 3, 2020

openjdk bot commented Dec 3, 2020

amitdpawar commented Nov 29, 2020 •

edited by openjdk bot

Loading

mlbridge bot commented Nov 29, 2020 •

edited

Loading

openjdk bot commented Dec 1, 2020 •

edited

Loading