Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8269571: NMT should print total malloc bytes and invocation count #4622

Conversation

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Jun 29, 2021

At the moment NMT prints a total of reserved and committed bytes. If one wants to know how much C-Heap the VM uses, one needs to add all numbers from the sub categories.

It would be helpful would NMT print the total number of malloced memory and additionally the malloc invocation count.


This patch modifes the printout to print, in addition to totals, the number of malloced bytes, malloc invocation counts, and number of mmap reserved/committed bytes. This mirrors the way we print individual sub categories and allocation sites.

Before:

Native Memory Tracking:

Total: reserved=18489703KB, committed=1183967KB

After:

Native Memory Tracking:

Total: reserved=18491726KB, committed=1183178KB
       malloc: 40590KB #13669
       mmap:   reserved=18451136KB, committed=1142588KB

Tests: Ran hotspot/runtime/NMT jtreg tests.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8269571: NMT should print total malloc bytes and invocation count

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4622/head:pull/4622
$ git checkout pull/4622

Update a local copy of the PR:
$ git checkout pull/4622
$ git pull https://git.openjdk.java.net/jdk pull/4622/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4622

View PR using the GUI difftool:
$ git pr show -t 4622

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4622.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Jun 29, 2021

👋 Welcome back stuefe! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

Loading

@openjdk openjdk bot added the rfr label Jun 29, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Jun 29, 2021

@tstuefe The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

Loading

@mlbridge
Copy link

@mlbridge mlbridge bot commented Jun 29, 2021

Webrevs

Loading

Copy link
Contributor

@zhengyu123 zhengyu123 left a comment

Good to me.

Loading

@openjdk
Copy link

@openjdk openjdk bot commented Jun 29, 2021

@tstuefe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8269571: NMT should print total malloc bytes and invocation count

Reviewed-by: zgu, xliu

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 8 new commits pushed to the master branch:

  • 1ac8dee: 8269615: Fix for 8263640 broke Windows build
  • 7ca753b: 8269268: JDWP: Properly fix thread lookup assert in findThread()
  • 7a23c9c: 8260540: serviceability/jdwp/AllModulesCommandTest.java failed with "Debuggee error: 'ERROR: transport error 202: bind failed: Address already in use'"
  • c1e2a29: 8263640: hs_err improvement: handle class path longer than O_BUFLEN
  • bb42d75: 8269417: Minor clarification on NonblockingQueue utility
  • e238cbd: 8269530: runtime/ParallelLoad/ParallelSuperTest.java timeout
  • 3f2c372: 8269126: Rename G1AllowPreventiveGC option to G1UsePreventiveGC
  • a977157: 8261579: AArch64: Support for weaker memory ordering in Atomic

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

Loading

@openjdk openjdk bot added the ready label Jun 29, 2021
@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Jun 29, 2021

Thank you @zhengyu123

Loading

@navyxliu
Copy link
Contributor

@navyxliu navyxliu commented Jun 29, 2021

your change looks good to me.

   malloc: 40590KB #13669

I think 13669 is the invocation count of malloc(). Out of curiosity, may I know how to use it? Do you intend to use this to pinpoint memleak from malloc()? A straight malloc() icount may not help much. Is it possible we can have the number of unbalanced malloc()? That is icount of malloc() minus icount of free(). of course, it is not directly related to this PR.

Loading

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Jun 30, 2021

your change looks good to me.

Thanks!

   malloc: 40590KB #13669

I think 13669 is the invocation count of malloc(). Out of curiosity, may I know how to use it? Do you intend to use this to pinpoint memleak from malloc()? A straight malloc() icount may not help much. Is it possible we can have the number of unbalanced malloc()? That is icount of malloc() minus icount of free(). of course, it is not directly related to this PR.

That count is not the invocation counter (which can be useful too under certain conditions) but the number of outstanding allocations.

Loading

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Jun 30, 2021

/integrate

Loading

@openjdk
Copy link

@openjdk openjdk bot commented Jun 30, 2021

Going to push as commit 3ad20fc.
Since your change was applied there have been 18 commits pushed to the master branch:

  • b969136: 8245877: assert(_value != __null) failed: resolving NULL _value in JvmtiExport::post_compiled_method_load
  • ee526a2: Merge
  • 0d745ae: 8269034: AccessControlException for SunPKCS11 daemon threads
  • d042029: 8269529: javax/swing/reliability/HangDuringStaticInitialization.java fails in Windows debug build
  • 401cb0a: 8269232: assert(!is_jweak(handle)) failed: wrong method for detroying jweak
  • b8a16e9: 8268884: C2: Compile::remove_speculative_types must iterate top-down
  • 25f9f19: 8249646: Runtime.exec(String, String[], File) documentation contains literal {@link ...}
  • 0d83dc2: 8268699: Shenandoah: Add test for JDK-8268127
  • 7010dfd: 8269517: compiler/loopopts/TestPartialPeelingSinkNodes.java crashes with -XX:+VerifyGraphEdges
  • fc0fd96: 8269126: Rename G1AllowPreventiveGC option to G1UsePreventiveGC
  • ... and 8 more: https://git.openjdk.java.net/jdk/compare/ee1e202bc36b8413e33b7b9e4c9f7a0601bf9a63...master

Your commit was automatically rebased without conflicts.

Loading

@openjdk openjdk bot closed this Jun 30, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Jun 30, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Jun 30, 2021

@tstuefe Pushed as commit 3ad20fc.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Loading

@navyxliu
Copy link
Contributor

@navyxliu navyxliu commented Jun 30, 2021

your change looks good to me.

Thanks!

   malloc: 40590KB #13669

I think 13669 is the invocation count of malloc(). Out of curiosity, may I know how to use it? Do you intend to use this to pinpoint memleak from malloc()? A straight malloc() icount may not help much. Is it possible we can have the number of unbalanced malloc()? That is icount of malloc() minus icount of free(). of course, it is not directly related to this PR.

That count is not the invocation counter (which can be useful too under certain conditions) but the number of outstanding allocations.
I see. I take a look at mallocTracker.hpp. It's unbalanced count.

Loading

@tstuefe tstuefe deleted the JDK-8269571-NMT-should-print-total-malloc-bytes-and-invocation-count branch Aug 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants