Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8262165: NMT report should state how many callsites had been skipped #2687

Conversation

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Feb 23, 2021

(This is a breakout from JDK-8261238, which seems to be stuck in review (#2428). I factor out the display changes, which hopefully are non-controversial enough to get upstream easily.)

NMT silently omits call sites in its report whose allocation rate is too small to get over certain, hidden, thresholds:
a) there is a hard wired threshold of 1K in the baseline collector
b) we also skip all sites whose size is less than what the unit is (eg scale=G, we skip all allocations < 1G

NMT report should state how many call sites had been omitted.


The patch adds a counter in the NMT reporter to count omitted call sites, then prints that counter out.

In base report, just a generic message is shown to notify the user that categories > scale are ignored:

thomas@mainframe$ jcmd Interl VM.native_memory scale=m
2847:

Native Memory Tracking:

(Omitting categories weighting less than 1MB)

Total: reserved=5540MB, committed=147MB
-                 Java Heap (reserved=3996MB, committed=34MB)
                            (mmap: reserved=3996MB, committed=34MB) 
<skip>

In detail report, the total count of omitted call sites is displayed at the bottom:

thomas@mainframe$ jcmd Interl VM.native_memory scale=m detail
2847:

<skip>

[0x00007f4e062e3e6b] os::reserve_memory(unsigned long, bool, MEMFLAGS)+0x16b
[0x00007f4e05ac4f47] MmapArrayAllocator<unsigned char>::allocate(unsigned long, MEMFLAGS)+0x127
[0x00007f4e05ac4cb2] G1FromCardCache::initialize(unsigned int)+0xe2
[0x00007f4e05a863f1] G1CollectedHeap::initialize()+0x3c1
                             (reserved=1MB, committed=1MB Type=GC)

(183 call sites weighting less than 1MB each omitted.)

Note: this patch only takes care of the sites omitted due to the scale threshold (b). It does not count how many sites had been omitted due to the baseline threshold (a). Since I still hope to remove that threshold completely with JDK-8261238, I did not put any work toward modifying the output here.


Tests: I manually ran runtime/NMT tests.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8262165: NMT report should state how many callsites had been skipped

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/2687/head:pull/2687
$ git checkout pull/2687

Update a local copy of the PR:
$ git checkout pull/2687
$ git pull https://git.openjdk.java.net/jdk pull/2687/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 2687

View PR using the GUI difftool:
$ git pr show -t 2687

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/2687.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Feb 23, 2021

👋 Welcome back stuefe! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

Loading

@openjdk
Copy link

@openjdk openjdk bot commented Feb 23, 2021

@tstuefe The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

Loading

@tstuefe tstuefe force-pushed the JDK-8262165-NMT-report-should-state-how-many-callsites-had-been-skipped branch from fa252e8 to a6bbf78 Feb 23, 2021
@tstuefe tstuefe marked this pull request as ready for review Feb 23, 2021
@openjdk openjdk bot added the rfr label Feb 23, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Feb 23, 2021

Webrevs

Loading

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Mar 6, 2021

Friendly Ping.

Loading

Copy link
Contributor

@zhengyu123 zhengyu123 left a comment

Looks good to me

Loading

@openjdk
Copy link

@openjdk openjdk bot commented Mar 8, 2021

@tstuefe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8262165: NMT report should state how many callsites had been skipped

Reviewed-by: zgu, shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

Loading

@openjdk openjdk bot added the ready label Mar 8, 2021
@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Mar 9, 2021

Looks good to me

Thanks Zhengyu!

Loading

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Mar 11, 2021

Can I have a second review please? Or could we, in this case, count my own pair of eyes as second reviewer, if no-one wants to look at NMT?

This change is really not that difficult.

Thanks, Thomas

Loading

Copy link
Contributor

@shipilev shipilev left a comment

Pretty simple, sure. Only minor nits.

Loading

}
// Don't report if site has reserved less than one unit of whatever our scale is
if (scale() > 1 && amount_in_current_scale(virtual_memory_site->reserved()) == 0) {
num_omitted ++;
Copy link
Contributor

@shipilev shipilev Apr 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the Hotspot style is to say num_omitted++.

Loading

void report_malloc_sites();
// Report virtual memory reservation sites
void report_virtual_memory_allocation_sites();
// Report malloc allocation sites; returns number of omitted sites.
Copy link
Contributor

@shipilev shipilev Apr 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and two lines below, comments do not need trailing points.

Loading

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Apr 16, 2021

Pretty simple, sure. Only minor nits.

Great, thanks, Aleksey. Good to have this finally off the table. I'll add the suggested modifications before pushing.

Loading

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented Apr 19, 2021

/integrate

Loading

@openjdk openjdk bot closed this Apr 19, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Apr 19, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Apr 19, 2021

@tstuefe Since your change was applied there have been 9 commits pushed to the master branch:

  • e390e55: 8265066: Split ReservedSpace constructor to avoid default parameter
  • c607d12: 8249528: Remove obsolete comment in G1RootProcessor::process_java_roots
  • fa58aae: 8265245: depChecker_ don't have any functionalities
  • a2b0e0f: 8265323: Leftover local variables in PcDesc
  • 1ac25b8: 8264372: Threads::destroy_vm only ever returns true
  • 73d5f3b: 8265313: Obsolete the unused AssertOnSuspendWaitFailure and TraceSuspendWaitFailures flags
  • cb8394a: 8265304: Temporarily make Metal the default 2D rendering pipeline for macOS
  • 66f8987: 8265298: Hard VM crash when deadlock between "access" and higher ranked lock is detected
  • 926e3bc: 8264143: Lanai: RenderPerfTest.BgrSwBlitImage has artefacts on apple M1

Your commit was automatically rebased without conflicts.

Pushed as commit 49b9e68.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Loading

@tstuefe tstuefe deleted the JDK-8262165-NMT-report-should-state-how-many-callsites-had-been-skipped branch Apr 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants