Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8281015: Further simplify NMT backend #7283

Closed
wants to merge 9 commits into from

Conversation

tstuefe
Copy link
Member

@tstuefe tstuefe commented Jan 31, 2022

NMT backend can be further simplified and cleaned out.

  • some entry points require NMT_TrackingLevel as arguments, some use the global tracking level. Ultimately, every part of NMT always uses the global tracking level, so in many cases the explicit parameter can be removed and the global tracking level can be used instead.
  • MemTracker::malloc_header_size(level) + MemTracker::malloc_footer_size(level) are fused into MemTracker::overhead_per_malloc()
  • when adding to MallocSiteTable, caller gets back a shortcut to the entry. That shortcut is stored verbatim in the malloc header. It consists of two 16-bit values (bucket index and chain position). That tupel finds its way into many argument lists. It can be simplified into single 32-bit opaque marker. Code outside the MallocSiteTable does not need to know what it is.
  • Currently, the MallocHeader class contains a lot of logic. It accounts (in constructor) and de-accounts (in MallocHeader::release()). It would simplify code if MallocHeader were just a dumb data carrier and the MallocTracker would do the actual work.
  • MallocHeader can be simplified, almost all members made constant and modifying accessors removed.
  • In some places we handle inputptr=NULL gracefully where we should assert instead
  • Expressions like MemTracker::tracking_level() != NMT_off can be simplified to MemTracker::enabled().
  • MemTracker::malloc_base (all variants) can be removed. Note that we have MallocTracker::malloc_header, which achieves the same and does not require casting to the header.

Testing:

  • GHAs
  • manually ran NMT gtests (all NMT modes) and NMT jtreg tests on Ubuntu x64
  • SAP nightlies ran through. Note that since 8275301 "Unify C-heap buffer overrun checks into NMT" NMT is enabled by default in debug builds, so it gets a lot more workout in tests now.

Note that I wanted to manually verify that the gdb "call pp" command still works in order to not break Zhengyu's recent addition, but found its already broken. I filed https://bugs.openjdk.java.net/browse/JDK-8281023 and am preparing a separate patch.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/7283/head:pull/7283
$ git checkout pull/7283

Update a local copy of the PR:
$ git checkout pull/7283
$ git pull https://git.openjdk.java.net/jdk pull/7283/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 7283

View PR using the GUI difftool:
$ git pr show -t 7283

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/7283.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Jan 31, 2022

👋 Welcome back stuefe! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Jan 31, 2022

@tstuefe The following label will be automatically applied to this pull request:

  • hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot hotspot-dev@openjdk.org label Jan 31, 2022
@tstuefe tstuefe changed the title NMT cleanups (wip) JDK-8281015: Further simplify NMT backend Feb 1, 2022
@tstuefe tstuefe marked this pull request as ready for review February 1, 2022 08:53
@openjdk openjdk bot added the rfr Pull request is ready for review label Feb 1, 2022
@mlbridge
Copy link

mlbridge bot commented Feb 1, 2022

Webrevs

@tstuefe
Copy link
Member Author

tstuefe commented Feb 15, 2022

Tested at SAP for 14 days, no problems.

Any opinions? Should I reduce this patch, or split it into parts to make it more palatable?

Copy link
Contributor

@zhengyu123 zhengyu123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall is good, a few minor comments.

// Never turned on
if (level == NMT_off || memblock == NULL) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wanna add assert memblock != NULL?

uint8_t _flags;
uint8_t _unused;
const size_t _size;
const uint32_t _mst_marker;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make mst_marker a struct? instead of opaque type.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I played around a lot with different forms (struct, union) and in the end settled on an opaque uint32 since

  • it would be passed by value and I would have to provide that structure in all kind places, I got include circularities
  • I have a vague improvement in my head where we store the malloc site table entries not as individually malloced elements but in a (dynamically growing) array; that would mean we could address them by index without having to walk the bucket chains; and that index would be a simple number.

@openjdk
Copy link

openjdk bot commented Feb 17, 2022

@tstuefe this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout nmt-cleanups
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

@openjdk openjdk bot added the merge-conflict Pull request has merge conflict with target branch label Feb 17, 2022
@tstuefe
Copy link
Member Author

tstuefe commented Feb 23, 2022

Thanks a lot, @zhengyu123, for your review. Sorry for the delay, I had vacation. I'll implement all your proposals excluding the last one (mst_marker as structure); see comment there.

@openjdk
Copy link

openjdk bot commented Feb 23, 2022

@tstuefe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8281015: Further simplify NMT backend

Reviewed-by: zgu, mbaesken

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 7 new commits pushed to the master branch:

  • 340a35d: 8282279: Interpret case-insensitive string locale independently
  • 5035bf5: 8282208: Reduce MachNode size
  • aaab2cb: 8282225: GHA: Allow one concurrent run per PR only
  • d017e98: 8255577: Possible issues with SR_initialize
  • 9332071: 8282194: C1: Missing side effects of dynamic constant linkage
  • ecd85e6: 8282231: x86-32: runtime call to SharedRuntime::ldiv corrupts registers
  • 378fa50: 8281962: Avoid unnecessary native calls in InflaterInputStream

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added ready Pull request is ready to be integrated and removed merge-conflict Pull request has merge conflict with target branch labels Feb 23, 2022
Copy link
Member

@MBaesken MBaesken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check copyright years, e.g. src/hotspot/share/services/memTracker.cpp (still 2021).
Otherwise looks okay to me.

@tstuefe
Copy link
Member Author

tstuefe commented Feb 23, 2022

please check copyright years, e.g. src/hotspot/share/services/memTracker.cpp (still 2021). Otherwise looks okay to me.

Thank you @MBaesken !

I will fix copyrights before pushing.

@tstuefe
Copy link
Member Author

tstuefe commented Feb 25, 2022

/integrate

@openjdk
Copy link

openjdk bot commented Feb 25, 2022

Going to push as commit b96b743.
Since your change was applied there have been 32 commits pushed to the master branch:

  • 9471f24: 8280684: JfrRecorderService failes with guarantee(num_written > 0) when no space left on device.
  • 3efd6aa: 8282347: AARCH64: Untaken branch in has_negatives stub
  • cd36be4: 8206187: javax/management/remote/mandatory/connection/DefaultAgentFilterTest.java fails with Port already in use
  • bf19fc6: 8280357: user.home = "?" when running with systemd DynamicUser=true
  • b6843a1: 8005885: enhance PrintCodeCache to print more data
  • 23995f8: 8281525: Enable Zc:strictStrings flag in Visual Studio build
  • 20e78f7: 8282307: Parallel: Incorrect discovery mode in PCReferenceProcessor
  • 0b6862e: 8282348: Remove unused CardTable::dirty_card_iterate
  • 6fab8a2: 8277204: Implement PAC-RET branch protection on Linux/AArch64
  • abc0ce1: 8282316: Operation before String case conversion
  • ... and 22 more: https://git.openjdk.java.net/jdk/compare/e1060bee2adb9f2e07ca09309d0f89132db30f28...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Feb 25, 2022
@openjdk openjdk bot closed this Feb 25, 2022
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Feb 25, 2022
@openjdk
Copy link

openjdk bot commented Feb 25, 2022

@tstuefe Pushed as commit b96b743.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@tstuefe tstuefe deleted the nmt-cleanups branch February 15, 2023 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot hotspot-dev@openjdk.org integrated Pull request has been integrated
3 participants