8264735: Make dynamic dump repeatable #4646

yminqi · 2021-07-01T02:39:07Z

Hi, Please review

Currently after dynamic dump, dump time tables (_dumptime_talbles, _dumptime_lambda_proxy_class_dictionary and _saved_shared_path_table) are corrupted and could not be used for next dump. The patch clones the three tables, and after dump restore them so the next dump is possible. With the fix, jcmd VM.cds dynamic_dump can do multiple dump to the same live process.

Tests: tier1,tier2,tier3,tier4

Thanks
Yumin

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8264735: Make dynamic dump repeatable

Reviewers

Calvin Cheung (@calvinccheung - Reviewer) ⚠️ Review applies to 9b64a16
Ioi Lam (@iklam - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4646/head:pull/4646
$ git checkout pull/4646

Update a local copy of the PR:
$ git checkout pull/4646
$ git pull https://git.openjdk.java.net/jdk pull/4646/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4646

View PR using the GUI difftool:
$ git pr show -t 4646

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4646.diff

bridgekeeper · 2021-07-01T02:40:24Z

👋 Welcome back minqi! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-07-01T02:42:02Z

@yminqi The following label will be automatically applied to this pull request:

hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2021-07-01T02:47:12Z

Webrevs

src/hotspot/share/cds/dumpTimeClassInfo.cpp

calvinccheung · 2021-07-02T17:58:59Z

src/hotspot/share/cds/dynamicArchive.cpp

@@ -158,6 +162,11 @@ class DynamicArchiveBuilder : public ArchiveBuilder {
    write_archive(serialized_data);
    release_header();

+    post_dump();
+
+    // Restore dumptime talbes


Typo: talbes -> tables

calvinccheung · 2021-07-02T17:59:09Z

src/hotspot/share/cds/filemap.cpp

+  assert(jrt != NULL,
+         "No modular java runtime image present when allocating the CDS classpath entry table");
+
+  // After dynamic dump, _saved_shared_path_table is corrupt, can not be used again.


Typo: can not -> cannot

Will update.

src/hotspot/share/cds/lambdaProxyClassDictionary.hpp

src/hotspot/share/classfile/systemDictionaryShared.cpp

calvinccheung · 2021-07-02T17:59:41Z

src/hotspot/share/classfile/systemDictionaryShared.cpp

+void SystemDictionaryShared::restore_dumptime_tables() {
+  // FileMapInfo::clean_cloned_shared_path_table();


Please delete line 1605 if it is not needed.

Ah, forget to delete it. Thanks

calvinccheung

Looks good. I have a few minor comments.

openjdk · 2021-07-06T16:14:29Z

@yminqi This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8264735: Make dynamic dump repeatable

Reviewed-by: ccheung, iklam

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 11 new commits pushed to the master branch:

d1cecaa: 8269481: SctpMultiChannel never releases own file descriptor
2209e3e: 8270027: ProblemList jdk/jfr/event/oldobject/TestObjectSize.java on macOS-x64
2dc5486: 8267303: Replace MinObjectAlignmentSize usages for non-Java heap objects
3586a23: 8268635: Corrupt oop in ClassLoaderData
bffb1a7: 8269923: runtime/jni/checked/TestPrimitiveArrayCriticalWithBadParam.java failed with "FATAL ERROR in native method: Primitive type array expected but not received for JNI array operation"
77a5b7b: 8269761: idea.sh missing .exe suffix when invoking javac on WSL
248aa50: 8269294: Verify_before/after_young_collection should execute all verification
18b80c7: 8269908: Move MemoryService::track_memory_usage call into G1MonitoringScope
a685011: 8269022: Put evacuation failure string directly into gc=info log message
72530ef: 8269574: C2: Avoid redundant uncommon traps in GraphKit::builtin_throw() for JVMTI exception events
... and 1 more: https://git.openjdk.java.net/jdk/compare/a9e201016de119af4b0fd3ebb43768896fb9e5c5...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

iklam · 2021-07-06T21:40:46Z

src/hotspot/share/cds/dynamicArchive.cpp

+    post_dump();
+
+    // Restore dumptime tables
+    SystemDictionaryShared::restore_dumptime_tables();


Maybe this should be put inside post_dump as well?

It would be better to keep it here with SystemDictionaryShared::clone_dumptime_table().

src/hotspot/share/cds/filemap.cpp

iklam · 2021-07-06T21:57:00Z

src/hotspot/share/cds/lambdaProxyClassDictionary.hpp

+  // GCC will memset for default ctor. With user defined ctor, data members need manually initialized.
+  // We could not use default ctor, it will destroy allocation_type in debug version and fail to delete.
+  // see bug 8269537
+  DumpTimeLambdaProxyClassDictionary() : _count(0)  {}


Since you changed the construction call from

(ResourceObj::C_HEAP, mtClass)DumpTimeLambdaProxyClassDictionary();

to

(ResourceObj::C_HEAP, mtClass)DumpTimeLambdaProxyClassDictionary;

I think it's no longer necessary to add the RunTimeLambdaProxyClassInfo() and DumpTimeSharedClassTable() constructors in this PR. In 8269537, we should find a generic way to prevent this bug from happening.

Initialization of _count is still needed. I could remove DumpTimeSharedClassTable, but still, we should keep the ctor which initialize the two data members --- though they will be reset in update_counts. I will remove the comments.

iklam

LGTM

yminqi · 2021-07-07T19:40:23Z

@calvinccheung @iklam Thanks for review!
/integrate

openjdk · 2021-07-07T19:42:01Z

Going to push as commit f741e4c.
Since your change was applied there have been 11 commits pushed to the master branch:

d1cecaa: 8269481: SctpMultiChannel never releases own file descriptor
2209e3e: 8270027: ProblemList jdk/jfr/event/oldobject/TestObjectSize.java on macOS-x64
2dc5486: 8267303: Replace MinObjectAlignmentSize usages for non-Java heap objects
3586a23: 8268635: Corrupt oop in ClassLoaderData
bffb1a7: 8269923: runtime/jni/checked/TestPrimitiveArrayCriticalWithBadParam.java failed with "FATAL ERROR in native method: Primitive type array expected but not received for JNI array operation"
77a5b7b: 8269761: idea.sh missing .exe suffix when invoking javac on WSL
248aa50: 8269294: Verify_before/after_young_collection should execute all verification
18b80c7: 8269908: Move MemoryService::track_memory_usage call into G1MonitoringScope
a685011: 8269022: Put evacuation failure string directly into gc=info log message
72530ef: 8269574: C2: Avoid redundant uncommon traps in GraphKit::builtin_throw() for JVMTI exception events
... and 1 more: https://git.openjdk.java.net/jdk/compare/a9e201016de119af4b0fd3ebb43768896fb9e5c5...master

Your commit was automatically rebased without conflicts.

openjdk · 2021-07-07T19:42:17Z

@yminqi Pushed as commit f741e4c.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8264735: Make dynamic dump repeatable

6245a80

openjdk bot added the rfr Pull request is ready for review label Jul 1, 2021

openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Jul 1, 2021

calvinccheung reviewed Jul 2, 2021

View reviewed changes

src/hotspot/share/cds/dumpTimeClassInfo.cpp Show resolved Hide resolved

calvinccheung reviewed Jul 2, 2021

View reviewed changes

src/hotspot/share/cds/lambdaProxyClassDictionary.hpp Show resolved Hide resolved

calvinccheung reviewed Jul 2, 2021

View reviewed changes

src/hotspot/share/classfile/systemDictionaryShared.cpp Show resolved Hide resolved

calvinccheung reviewed Jul 2, 2021

View reviewed changes

calvinccheung suggested changes Jul 2, 2021

View reviewed changes

Fix comments, added check holding for DumpTimeTable_lock

9b64a16

calvinccheung approved these changes Jul 6, 2021

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Jul 6, 2021

iklam suggested changes Jul 6, 2021

View reviewed changes

Yumin Qi added 2 commits July 6, 2021 17:39

Merge branch 'master' into jdk-8264735

3282410

Save SharedPathTable array for later reclaim, remove comments for ctors

28dfcaf

iklam approved these changes Jul 7, 2021

View reviewed changes

openjdk bot closed this Jul 7, 2021

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jul 7, 2021

yminqi deleted the jdk-8264735 branch July 7, 2021 19:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8264735: Make dynamic dump repeatable #4646

8264735: Make dynamic dump repeatable #4646

yminqi commented Jul 1, 2021 •

edited by openjdk bot

Loading

bridgekeeper bot commented Jul 1, 2021

openjdk bot commented Jul 1, 2021

mlbridge bot commented Jul 1, 2021 •

edited

Loading

calvinccheung Jul 2, 2021

calvinccheung Jul 2, 2021

yminqi Jul 2, 2021

calvinccheung Jul 2, 2021

yminqi Jul 2, 2021

calvinccheung left a comment

openjdk bot commented Jul 6, 2021 •

edited

Loading

iklam Jul 6, 2021

yminqi Jul 6, 2021

iklam Jul 6, 2021

yminqi Jul 7, 2021

iklam left a comment

yminqi commented Jul 7, 2021

openjdk bot commented Jul 7, 2021

openjdk bot commented Jul 7, 2021

		void SystemDictionaryShared::restore_dumptime_tables() {
		// FileMapInfo::clean_cloned_shared_path_table();

8264735: Make dynamic dump repeatable #4646

8264735: Make dynamic dump repeatable #4646

Conversation

yminqi commented Jul 1, 2021 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Jul 1, 2021

openjdk bot commented Jul 1, 2021

mlbridge bot commented Jul 1, 2021 • edited Loading

Webrevs

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

calvinccheung left a comment

Choose a reason for hiding this comment

openjdk bot commented Jul 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iklam left a comment

Choose a reason for hiding this comment

yminqi commented Jul 7, 2021

openjdk bot commented Jul 7, 2021

openjdk bot commented Jul 7, 2021

yminqi commented Jul 1, 2021 •

edited by openjdk bot

Loading

mlbridge bot commented Jul 1, 2021 •

edited

Loading

openjdk bot commented Jul 6, 2021 •

edited

Loading