Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8277990: NMT: Remove NMT shutdown capability #6640

Closed

Conversation

zhengyu123
Copy link
Contributor

@zhengyu123 zhengyu123 commented Dec 1, 2021

NMT shutdown functionality is a remnant of its first implementation, which could consume excessive amount of memory, therefore, it needed capability to shut it self down to ensure health of JVM. This is no longer a case for current implementation.

After JDK-8277946, there is no longer a use, so it can be removed.

Test:

  • hotspot_nmt
  • tier1 with NMT on

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/6640/head:pull/6640
$ git checkout pull/6640

Update a local copy of the PR:
$ git checkout pull/6640
$ git pull https://git.openjdk.java.net/jdk pull/6640/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 6640

View PR using the GUI difftool:
$ git pr show -t 6640

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/6640.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Dec 1, 2021

👋 Welcome back zgu! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Dec 1, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Dec 1, 2021

@zhengyu123 The following label will be automatically applied to this pull request:

  • hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot label Dec 1, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 1, 2021

Webrevs

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Dec 2, 2021

Very nice simplification.

Before diving in, could you explain a bit the locking changes in MallocSiteTable? To me, the association with shutdown is not immediately clear. Thanks!

@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Dec 2, 2021

Very nice simplification.

Before diving in, could you explain a bit the locking changes in MallocSiteTable? To me, the association with shutdown is not immediately clear. Thanks!

You are talking about AccessLock, right?

As @dholmes-ora mentioned in PR #6267, the name is misleading: it is not a lock, but a countdown latch. It allows multi-reader to access MallocSiteTable, but once an exclusive access is requested, the requester sets counter to negative number and waits all readers to exit, then no readers and writers are allowed, so that MallocSiteTable can be safely destroyed.

AccessLock was invented to guard MallocSiteTable, as ThreadCritical is too expensive for malloc tracking.

@dcubed-ojdk
Copy link
Member

@dcubed-ojdk dcubed-ojdk commented Dec 2, 2021

Hmmm... sounds like a ReadersWriter lock to me... Any number of Readers and a single
Writer after all the Readers have released... It has surprised me before that HotSpot does
not have such a useful mechanism...

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Dec 2, 2021

You are talking about AccessLock, right?

As @dholmes-ora mentioned in PR #6267, the name is misleading: it is not a lock, but a countdown latch. It allows multi-reader to access MallocSiteTable, but once an exclusive access is requested, the requester sets counter to negative number and waits all readers to exit, then no readers and writers are allowed, so that MallocSiteTable can be safely destroyed.

AccessLock was invented to guard MallocSiteTable, as ThreadCritical is too expensive for malloc tracking.

Just to see I get it right, AccessLock is not needed anymore because we don't destroy the MallocSiteTable? We don't need it to guard the table when we add callstacks?

@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Dec 2, 2021

Hmmm... sounds like a ReadersWriter lock to me... Any number of Readers and a single Writer after all the Readers have released... It has surprised me before that HotSpot does not have such a useful mechanism...

Well, the difference is that, it does not have well defined critical section. It depends on other structures/operations for data integrity.

@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Dec 2, 2021

You are talking about AccessLock, right?

As @dholmes-ora mentioned in PR #6267, the name is misleading: it is not a lock, but a countdown latch. It allows multi-reader to access MallocSiteTable, but once an exclusive access is requested, the requester sets counter to negative number and waits all readers to exit, then no readers and writers are allowed, so that MallocSiteTable can be safely destroyed.

AccessLock was invented to guard MallocSiteTable, as ThreadCritical is too expensive for malloc tracking.

Just to see I get it right, AccessLock is not needed anymore because we don't destroy the MallocSiteTable? We don't need it to guard the table when we add callstacks?

Yes. Ironically, adding entry only requires shared access, it uses CAS to ensure table integrity :-) If we don't destroy the table, there is no requester for exclusive access, so it is no longer needed.

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Dec 2, 2021

Yes. Ironically, adding entry only requires shared access, it uses CAS to ensure table integrity :-) If we don't destroy the table, there is no requester for exclusive access, so it is no longer needed.

I see. That's nice. Thanks for the clarification.

Copy link
Member

@tstuefe tstuefe left a comment

Nice cleanup.

You missed a few:

  • test/lib/jdk/test/whitebox/WhiteBox.java, NMTChangeTrackingLevel
  • test/lib/sun/hotspot/WhiteBox.java, same

Will the removal of WB functions cause incompatibilities with older versions of jtreg (can't imagine it does, just wondering)?

Cheers, Thomas

src/hotspot/os/posix/perfMemory_posix.cpp Outdated Show resolved Hide resolved
src/hotspot/share/services/nmtCommon.hpp Show resolved Hide resolved
src/hotspot/share/services/nmtCommon.hpp Show resolved Hide resolved
@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Dec 2, 2021

Nice cleanup.

You missed a few:

  • test/lib/jdk/test/whitebox/WhiteBox.java, NMTChangeTrackingLevel
  • test/lib/sun/hotspot/WhiteBox.java, same

Out of curious, why have two versions?

Will the removal of WB functions cause incompatibilities with older versions of jtreg (can't imagine it does, just wondering)?

No idea.

Cheers, Thomas

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Dec 2, 2021

Nice cleanup.
You missed a few:

  • test/lib/jdk/test/whitebox/WhiteBox.java, NMTChangeTrackingLevel
  • test/lib/sun/hotspot/WhiteBox.java, same

Out of curious, why have two versions?

No clue :)

tstuefe
tstuefe approved these changes Dec 2, 2021
Copy link
Member

@tstuefe tstuefe left a comment

Looks good to me. Thank you for doing this!

..Thomas

@openjdk
Copy link

@openjdk openjdk bot commented Dec 2, 2021

@zhengyu123 This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8277990: NMT: Remove NMT shutdown capability

Reviewed-by: stuefe, shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 107 new commits pushed to the master branch:

  • bd7c54a: 8278341: Liveness check for global scope is not as fast as it could be
  • c609b5d: 8277628: Spec for InetAddressResolverProvider::get() throwing error or exception could be clearer
  • bb50b92: 8277536: Use String.blank in jdk.javadoc where applicable
  • 5b81d5e: 8276901: Implement UseHeavyMonitors consistently
  • 69d8669: 8278339: ServerSocket::isClosed may return false after accept throws
  • 56ca66e: 8277863: Deprecate sun.misc.Unsafe methods that return offsets
  • 3536127: 8277383: VM.metaspace optionally show chunk freelist details
  • 44fcee3: 8278289: Drop G1BlockOffsetTablePart::_object_can_span
  • b2638e5: 8244602: Add JTREG_REPEAT_COUNT to repeat execution of a test
  • 07669e3: 8275375: [REDO] JDK-8271949 dumppath in -XX:FlightRecorderOptions does not affect
  • ... and 97 more: https://git.openjdk.java.net/jdk/compare/abaa073bcbdb202658c8a97401ffb098d71e0f16...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Dec 2, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 3, 2021

Mailing list message from David Holmes on hotspot-dev:

On 3/12/2021 4:23 am, Zhengyu Gu wrote:

On Thu, 2 Dec 2021 16:23:32 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

Nice cleanup.

You missed a few:

* test/lib/jdk/test/whitebox/WhiteBox.java, `NMTChangeTrackingLevel`
* test/lib/sun/hotspot/WhiteBox.java, same

Out of curious, why have two versions?

Historical. WB is primarily a hotspot test aid and resided in the
hotspot testlibrary in the hotspot repo. Overtime we've been moving to a
common testlibrary for everyone now we are all in one repo. No idea
where we are with that consolidation process though. I think the person
driving that may no longer be around, not sure.

Will the removal of WB functions cause incompatibilities with older versions of jtreg (can't imagine it does, just wondering)?

If you mean in terms of @require checking that involves WB I believe
that support class is part of the repo not jtreg.

Cheers,
David
-----

@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Dec 3, 2021

Looks good to me. Thank you for doing this!

..Thomas

Thanks for the review, @tstuefe

Copy link
Contributor

@shipilev shipilev left a comment

Very nice cleanup. Some minor suggestions below, feel free to ignore them.

@@ -281,7 +260,7 @@ void* MallocTracker::record_malloc(void* malloc_base, size_t size, MEMFLAGS flag
assert(((size_t)memblock & (sizeof(size_t) * 2 - 1)) == 0, "Alignment check");

#ifdef ASSERT
if (level > NMT_minimal) {
if (level >= NMT_summary) {
Copy link
Contributor

@shipilev shipilev Dec 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (level >= NMT_summary) {
if (level > NMT_off) {

This way, we don't care where NMT_summary is in the enum. We enter this code on all paths when NMT is enabled?

Copy link
Contributor

@shipilev shipilev Dec 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, maybe even call to enabled() here?

NMT_TrackingLevel level = tracking_level();
if (level >= NMT_summary) {
report(level == NMT_summary, output, 1);
if (enabled()) {
Copy link
Contributor

@shipilev shipilev Dec 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks to me, you can do enabled() && Atomic::cmpxchg(...), thus saving a tiny bit of CPU cycles when NMT is disabled?

// Transition the tracking level to specified level
static bool transition_to(NMT_TrackingLevel level);
static inline bool enabled() {
return _tracking_level >= NMT_summary;
Copy link
Contributor

@shipilev shipilev Dec 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return _tracking_level >= NMT_summary;
return _tracking_level > NMT_off;

Same reason as above.

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Dec 7, 2021

Let's ship this, I'm really interested in getting this into JDK 18 before ramp-down on Thursday.

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Dec 7, 2021

Patch is still good after latest changes.

Copy link
Contributor

@shipilev shipilev left a comment

Good. Yes, let's try to make it into JDK 18.

@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Dec 7, 2021

Thanks, @tstuefe @shipilev. Will integrate once GHA is clean

@zhengyu123
Copy link
Contributor Author

@zhengyu123 zhengyu123 commented Dec 7, 2021

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Dec 7, 2021

Going to push as commit 5a036ac.
Since your change was applied there have been 110 commits pushed to the master branch:

  • 7217cb7: 8274883: (se) Selector.open throws IAE when the default file system provider is changed to a custom provider
  • 7ea4b19: 8278166: java/nio/channels/Channels/TransferTo.java timed out
  • 543d1a8: 8275721: Name of UTC timezone in a locale changes depending on previous code
  • bd7c54a: 8278341: Liveness check for global scope is not as fast as it could be
  • c609b5d: 8277628: Spec for InetAddressResolverProvider::get() throwing error or exception could be clearer
  • bb50b92: 8277536: Use String.blank in jdk.javadoc where applicable
  • 5b81d5e: 8276901: Implement UseHeavyMonitors consistently
  • 69d8669: 8278339: ServerSocket::isClosed may return false after accept throws
  • 56ca66e: 8277863: Deprecate sun.misc.Unsafe methods that return offsets
  • 3536127: 8277383: VM.metaspace optionally show chunk freelist details
  • ... and 100 more: https://git.openjdk.java.net/jdk/compare/abaa073bcbdb202658c8a97401ffb098d71e0f16...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Dec 7, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Dec 7, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Dec 7, 2021

@zhengyu123 Pushed as commit 5a036ac.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot integrated
4 participants