8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum #645

yminqi · 2020-10-13T23:40:54Z

Please review this simple change.
Snapshot should be done first on NonClassType which will include ClassType allocation when Metaspace::using_class_space() is false. _class_vsm (a VirtualSpaceManager) is set only for class type allocation when using_class_space is true. The correct order is do snapshot for NonClassType first, then for ClassType if using_class_space.

Tests: mach5 tier1-4 in progress.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum

Reviewers

Thomas Stuefe (@tstuefe - Reviewer)
Richard Reingruber (@reinrich - Committer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/645/head:pull/645
$ git checkout pull/645

bridgekeeper · 2020-10-13T23:43:17Z

👋 Welcome back minqi! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-10-13T23:49:40Z

@yminqi The following label will be automatically applied to this pull request:

hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2020-10-13T23:52:18Z

Webrevs

00: Full (59eb51b)

iklam · 2020-10-14T01:07:30Z

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

yminqi · 2020-10-14T03:27:30Z

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

MetaspaceSnapshot predefine the _reserved_in_bytes (and other two, _commited_in_byte, _used_in_bytes) with dimension of MetadataTypeCount, the array of index ClassType is a valid slot. In this bug, if CCP is disabled, the snapshot for ClassType gets updated with zero, but non class type data is not update in snapshot. There is no output for class type data when CCP is disabled. While the original data recording is correct (at allocation) the snapshot did not get it. I will do some tests.

tstuefe · 2020-10-14T07:26:26Z

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

MetaspaceSnapshot predefine the _reserved_in_bytes (and other two, _commited_in_byte, _used_in_bytes) with dimension of MetadataTypeCount, the array of index ClassType is a valid slot. In this bug, if CCP is disabled, the snapshot for ClassType gets updated with zero, but non class type data is not update in snapshot. There is no output for class type data when CCP is disabled. While the original data recording is correct (at allocation) the snapshot did not get it. I will do some tests.

This is confusing to be sure. The part that is broken by this bug is using the basline..diff functionality if compressed class space is off. All other metaspace numbers are still okay because they are printed via other code paths.

Reproduction:

>java -XX:-UseCompressedClassPointers -XX:NativeMemoryTracking=summary de.stuefe.repros.metaspace.JustLoadAndLoad &
> jcmd JustLoadAndLoad VM.native_memory baseline
7518:
Baseline succeeded
> jcmd JustLoadAndLoad VM.native_memory summary.diff
7518:

Native Memory Tracking:

Total: reserved=5015640KB +163200KB, committed=543088KB +161924KB
<skip>
-                     Class (reserved=427146KB +161234KB, committed=425458KB +159954KB)
                            (classes #6011 +1185)
                            (  instance classes #5706 +1185, array classes #305)
                            (malloc=9354KB +3538KB #41433 +14309)
                            (mmap: reserved=417792KB +157696KB, committed=416104KB +156416KB)
                            (  Metadata:   )   << no increase shown
                            (    reserved=0KB, committed=0KB)
                            (    used=0KB)
                            (    free=0KB)
                            (    waste=0KB =-nan%)

And no, there are no tests for that. This is an old bug.

tstuefe

Looks good, thanks for fixing.

openjdk · 2020-10-14T07:27:46Z

@yminqi This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum

Reviewed-by: stuefe, rrich

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 14 new commits pushed to the master branch:

96a1f08: 8254775: Microbenchmark StringIndexOfChar doesn't compile
a6a3813: 8254761: Wrong intrinsic annotation used for StringLatin1.indexOfChar
738effa: 8254744: Clean-up CodeBlob::align_code_offset
f71e8a6: 8173585: Intrinsify StringLatin1.indexOf(char)
95e68c6: 8254586: Replace fthrow() calls with simpler method calls in classFileParser.cpp
ba140b0: 8254748: Bad Copyright header format after JDK-8212218
db9dcdf: 8254668: JVMTI process frames on thread without started processing
dc262df: 8212218: [TESTBUG] runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java timed out
9eeeb8a: 8254696: safepointMechanism_aix needs adaptation for JDK-8253180
b509e31: 8254602: compiler/debug/TestStressCM.java failed with "RuntimeException: got the same optimization stats for different seeds: expected 45"
... and 4 more: https://git.openjdk.java.net/jdk/compare/ba5dc67a74c35e64b934ec8105a853770b30ed52...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

reinrich

The change looks good // not Reviewer though.
Thanks for fixing!

tstuefe · 2020-10-14T10:49:10Z

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

MetaspaceSnapshot predefine the _reserved_in_bytes (and other two, _commited_in_byte, _used_in_bytes) with dimension of MetadataTypeCount, the array of index ClassType is a valid slot. In this bug, if CCP is disabled, the snapshot for ClassType gets updated with zero, but non class type data is not update in snapshot. There is no output for class type data when CCP is disabled. While the original data recording is correct (at allocation) the snapshot did not get it. I will do some tests.

This is confusing to be sure. The part that is broken by this bug is using the basline..diff functionality if compressed class space is off. All other metaspace numbers are still okay because they are printed via other code paths.

Reproduction:
>java -XX:-UseCompressedClassPointers -XX:NativeMemoryTracking=summary de.stuefe.repros.metaspace.JustLoadAndLoad &
> jcmd JustLoadAndLoad VM.native_memory baseline
7518:
Baseline succeeded
> jcmd JustLoadAndLoad VM.native_memory summary.diff
7518:

Native Memory Tracking:

Total: reserved=5015640KB +163200KB, committed=543088KB +161924KB
<skip>
-                     Class (reserved=427146KB +161234KB, committed=425458KB +159954KB)
                            (classes #6011 +1185)
                            (  instance classes #5706 +1185, array classes #305)
                            (malloc=9354KB +3538KB #41433 +14309)
                            (mmap: reserved=417792KB +157696KB, committed=416104KB +156416KB)
                            (  Metadata:   )   << no increase shown
                            (    reserved=0KB, committed=0KB)
                            (    used=0KB)
                            (    free=0KB)
                            (    waste=0KB =-nan%)
And no, there are no tests for that. This is an old bug.

Not sure how Oracle handles this usually, but maybe it makes sense to reformulate the JBS issue and describe above effect? Or, alternatively, open a new one as a duplicate to this bug?

When people look into the JBS for this specific NMT bug they may not notice the harmless test error.

yminqi · 2020-10-14T16:28:22Z

@tstuefe @reinrich
Thanks for review!
I am not sure if JBS issue changed what the PR will react --- Skara requests exact match for the issue. File a new bug with correct description as a dup of this one is OK I think.

yminqi · 2020-10-14T17:11:01Z

/integrate

openjdk · 2020-10-14T17:13:44Z

@yminqi Since your change was applied there have been 14 commits pushed to the master branch:

96a1f08: 8254775: Microbenchmark StringIndexOfChar doesn't compile
a6a3813: 8254761: Wrong intrinsic annotation used for StringLatin1.indexOfChar
738effa: 8254744: Clean-up CodeBlob::align_code_offset
f71e8a6: 8173585: Intrinsify StringLatin1.indexOf(char)
95e68c6: 8254586: Replace fthrow() calls with simpler method calls in classFileParser.cpp
ba140b0: 8254748: Bad Copyright header format after JDK-8212218
db9dcdf: 8254668: JVMTI process frames on thread without started processing
dc262df: 8212218: [TESTBUG] runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java timed out
9eeeb8a: 8254696: safepointMechanism_aix needs adaptation for JDK-8253180
b509e31: 8254602: compiler/debug/TestStressCM.java failed with "RuntimeException: got the same optimization stats for different seeds: expected 45"
... and 4 more: https://git.openjdk.java.net/jdk/compare/ba5dc67a74c35e64b934ec8105a853770b30ed52...master

Your commit was automatically rebased without conflicts.

Pushed as commit fde02e2.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum

59eb51b

openjdk bot added the rfr Pull request is ready for review label Oct 13, 2020

openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Oct 13, 2020

tstuefe approved these changes Oct 14, 2020

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 14, 2020

reinrich approved these changes Oct 14, 2020

View reviewed changes

openjdk bot closed this Oct 14, 2020

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 14, 2020

yminqi deleted the jdk-8254012 branch October 14, 2020 17:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum #645

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum #645

yminqi commented Oct 13, 2020 •

edited by openjdk bot

Loading

bridgekeeper bot commented Oct 13, 2020

openjdk bot commented Oct 13, 2020

mlbridge bot commented Oct 13, 2020

iklam commented Oct 14, 2020

yminqi commented Oct 14, 2020 •

edited

Loading

tstuefe commented Oct 14, 2020

tstuefe left a comment

openjdk bot commented Oct 14, 2020 •

edited

Loading

reinrich left a comment

tstuefe commented Oct 14, 2020

yminqi commented Oct 14, 2020

yminqi commented Oct 14, 2020

openjdk bot commented Oct 14, 2020

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum #645

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum #645

Conversation

yminqi commented Oct 13, 2020 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Download

bridgekeeper bot commented Oct 13, 2020

openjdk bot commented Oct 13, 2020

mlbridge bot commented Oct 13, 2020

Webrevs

iklam commented Oct 14, 2020

yminqi commented Oct 14, 2020 • edited Loading

tstuefe commented Oct 14, 2020

tstuefe left a comment

Choose a reason for hiding this comment

openjdk bot commented Oct 14, 2020 • edited Loading

reinrich left a comment

Choose a reason for hiding this comment

tstuefe commented Oct 14, 2020

yminqi commented Oct 14, 2020

yminqi commented Oct 14, 2020

openjdk bot commented Oct 14, 2020

yminqi commented Oct 13, 2020 •

edited by openjdk bot

Loading

yminqi commented Oct 14, 2020 •

edited

Loading

openjdk bot commented Oct 14, 2020 •

edited

Loading