Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum #645

Closed
wants to merge 1 commit into from

Conversation

yminqi
Copy link
Contributor

@yminqi yminqi commented Oct 13, 2020

Please review this simple change.
Snapshot should be done first on NonClassType which will include ClassType allocation when Metaspace::using_class_space() is false. _class_vsm (a VirtualSpaceManager) is set only for class type allocation when using_class_space is true. The correct order is do snapshot for NonClassType first, then for ClassType if using_class_space.

Tests: mach5 tier1-4 in progress.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/645/head:pull/645
$ git checkout pull/645

@bridgekeeper
Copy link

bridgekeeper bot commented Oct 13, 2020

👋 Welcome back minqi! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Oct 13, 2020
@openjdk
Copy link

openjdk bot commented Oct 13, 2020

@yminqi The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Oct 13, 2020
@mlbridge
Copy link

mlbridge bot commented Oct 13, 2020

Webrevs

@iklam
Copy link
Member

iklam commented Oct 14, 2020

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

@yminqi
Copy link
Contributor Author

yminqi commented Oct 14, 2020

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

MetaspaceSnapshot predefine the _reserved_in_bytes (and other two, _commited_in_byte, _used_in_bytes) with dimension of MetadataTypeCount, the array of index ClassType is a valid slot. In this bug, if CCP is disabled, the snapshot for ClassType gets updated with zero, but non class type data is not update in snapshot. There is no output for class type data when CCP is disabled. While the original data recording is correct (at allocation) the snapshot did not get it. I will do some tests.

@tstuefe
Copy link
Member

tstuefe commented Oct 14, 2020

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

MetaspaceSnapshot predefine the _reserved_in_bytes (and other two, _commited_in_byte, _used_in_bytes) with dimension of MetadataTypeCount, the array of index ClassType is a valid slot. In this bug, if CCP is disabled, the snapshot for ClassType gets updated with zero, but non class type data is not update in snapshot. There is no output for class type data when CCP is disabled. While the original data recording is correct (at allocation) the snapshot did not get it. I will do some tests.

This is confusing to be sure. The part that is broken by this bug is using the basline..diff functionality if compressed class space is off. All other metaspace numbers are still okay because they are printed via other code paths.

Reproduction:

>java -XX:-UseCompressedClassPointers -XX:NativeMemoryTracking=summary de.stuefe.repros.metaspace.JustLoadAndLoad &
> jcmd JustLoadAndLoad VM.native_memory baseline
7518:
Baseline succeeded
> jcmd JustLoadAndLoad VM.native_memory summary.diff
7518:

Native Memory Tracking:

Total: reserved=5015640KB +163200KB, committed=543088KB +161924KB
<skip>
-                     Class (reserved=427146KB +161234KB, committed=425458KB +159954KB)
                            (classes #6011 +1185)
                            (  instance classes #5706 +1185, array classes #305)
                            (malloc=9354KB +3538KB #41433 +14309)
                            (mmap: reserved=417792KB +157696KB, committed=416104KB +156416KB)
                            (  Metadata:   )   << no increase shown
                            (    reserved=0KB, committed=0KB)
                            (    used=0KB)
                            (    free=0KB)
                            (    waste=0KB =-nan%)

And no, there are no tests for that. This is an old bug.

Copy link
Member

@tstuefe tstuefe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for fixing.

@openjdk
Copy link

openjdk bot commented Oct 14, 2020

@yminqi This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254012: NMT: MetaspaceSnapshot::snapshot uses wrong enum

Reviewed-by: stuefe, rrich

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 14 new commits pushed to the master branch:

  • 96a1f08: 8254775: Microbenchmark StringIndexOfChar doesn't compile
  • a6a3813: 8254761: Wrong intrinsic annotation used for StringLatin1.indexOfChar
  • 738effa: 8254744: Clean-up CodeBlob::align_code_offset
  • f71e8a6: 8173585: Intrinsify StringLatin1.indexOf(char)
  • 95e68c6: 8254586: Replace fthrow() calls with simpler method calls in classFileParser.cpp
  • ba140b0: 8254748: Bad Copyright header format after JDK-8212218
  • db9dcdf: 8254668: JVMTI process frames on thread without started processing
  • dc262df: 8212218: [TESTBUG] runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java timed out
  • 9eeeb8a: 8254696: safepointMechanism_aix needs adaptation for JDK-8253180
  • b509e31: 8254602: compiler/debug/TestStressCM.java failed with "RuntimeException: got the same optimization stats for different seeds: expected 45"
  • ... and 4 more: https://git.openjdk.java.net/jdk/compare/ba5dc67a74c35e64b934ec8105a853770b30ed52...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Oct 14, 2020
Copy link
Member

@reinrich reinrich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good // not Reviewer though.
Thanks for fixing!

@tstuefe
Copy link
Member

tstuefe commented Oct 14, 2020

I wonder why this bug has not been discovered until now. It seems like the old code would not work at all when compressed class space is disabled. Do we have any test cases for it? Or, is this feature actually used?

MetaspaceSnapshot predefine the _reserved_in_bytes (and other two, _commited_in_byte, _used_in_bytes) with dimension of MetadataTypeCount, the array of index ClassType is a valid slot. In this bug, if CCP is disabled, the snapshot for ClassType gets updated with zero, but non class type data is not update in snapshot. There is no output for class type data when CCP is disabled. While the original data recording is correct (at allocation) the snapshot did not get it. I will do some tests.

This is confusing to be sure. The part that is broken by this bug is using the basline..diff functionality if compressed class space is off. All other metaspace numbers are still okay because they are printed via other code paths.

Reproduction:

>java -XX:-UseCompressedClassPointers -XX:NativeMemoryTracking=summary de.stuefe.repros.metaspace.JustLoadAndLoad &
> jcmd JustLoadAndLoad VM.native_memory baseline
7518:
Baseline succeeded
> jcmd JustLoadAndLoad VM.native_memory summary.diff
7518:

Native Memory Tracking:

Total: reserved=5015640KB +163200KB, committed=543088KB +161924KB
<skip>
-                     Class (reserved=427146KB +161234KB, committed=425458KB +159954KB)
                            (classes #6011 +1185)
                            (  instance classes #5706 +1185, array classes #305)
                            (malloc=9354KB +3538KB #41433 +14309)
                            (mmap: reserved=417792KB +157696KB, committed=416104KB +156416KB)
                            (  Metadata:   )   << no increase shown
                            (    reserved=0KB, committed=0KB)
                            (    used=0KB)
                            (    free=0KB)
                            (    waste=0KB =-nan%)

And no, there are no tests for that. This is an old bug.

Not sure how Oracle handles this usually, but maybe it makes sense to reformulate the JBS issue and describe above effect? Or, alternatively, open a new one as a duplicate to this bug?

When people look into the JBS for this specific NMT bug they may not notice the harmless test error.

@yminqi
Copy link
Contributor Author

yminqi commented Oct 14, 2020

@tstuefe @reinrich
Thanks for review!
I am not sure if JBS issue changed what the PR will react --- Skara requests exact match for the issue. File a new bug with correct description as a dup of this one is OK I think.

@yminqi
Copy link
Contributor Author

yminqi commented Oct 14, 2020

/integrate

@openjdk openjdk bot closed this Oct 14, 2020
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 14, 2020
@openjdk
Copy link

openjdk bot commented Oct 14, 2020

@yminqi Since your change was applied there have been 14 commits pushed to the master branch:

  • 96a1f08: 8254775: Microbenchmark StringIndexOfChar doesn't compile
  • a6a3813: 8254761: Wrong intrinsic annotation used for StringLatin1.indexOfChar
  • 738effa: 8254744: Clean-up CodeBlob::align_code_offset
  • f71e8a6: 8173585: Intrinsify StringLatin1.indexOf(char)
  • 95e68c6: 8254586: Replace fthrow() calls with simpler method calls in classFileParser.cpp
  • ba140b0: 8254748: Bad Copyright header format after JDK-8212218
  • db9dcdf: 8254668: JVMTI process frames on thread without started processing
  • dc262df: 8212218: [TESTBUG] runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryErrorInMetaspace.java timed out
  • 9eeeb8a: 8254696: safepointMechanism_aix needs adaptation for JDK-8253180
  • b509e31: 8254602: compiler/debug/TestStressCM.java failed with "RuntimeException: got the same optimization stats for different seeds: expected 45"
  • ... and 4 more: https://git.openjdk.java.net/jdk/compare/ba5dc67a74c35e64b934ec8105a853770b30ed52...master

Your commit was automatically rebased without conflicts.

Pushed as commit fde02e2.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@yminqi yminqi deleted the jdk-8254012 branch Oct 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime hotspot-runtime-dev@openjdk.org integrated Pull request has been integrated
4 participants