Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8261034: improve jcmd GC.class_histogram to support parallel #2379

Closed
wants to merge 8 commits into from

Conversation

Hamlin-Li
Copy link

@Hamlin-Li Hamlin-Li commented Feb 3, 2021

parallel -histo of jmap was added by JDK-8214535, it's better to support parallel for jcmd GC.class_histogram too.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8261034: improve jcmd GC.class_histogram to support parallel

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/2379/head:pull/2379
$ git checkout pull/2379

Update a local copy of the PR:
$ git checkout pull/2379
$ git pull https://git.openjdk.java.net/jdk pull/2379/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 2379

View PR using the GUI difftool:
$ git pr show -t 2379

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/2379.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Feb 3, 2021

👋 Welcome back mli! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Feb 3, 2021
@openjdk
Copy link

openjdk bot commented Feb 3, 2021

@Hamlin-Li The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Feb 3, 2021
@mlbridge
Copy link

mlbridge bot commented Feb 3, 2021

Copy link
Contributor

@plummercj plummercj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'll need a CSR just as JDK-8215624. Also, a test is needed. There are a couple of tests under test/hotspot/jtreg/serviceability/dcmd/gc that already test GC.class_histogram. You could probably clone or modify one of them to test the new -parallel option.

Comment on lines 558 to 559
_parallel_thread_num("-parallel", "parallel threads number for heap iteration",
"INT", false, "0") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "0" mean use a default number of parallel threads as it does for the jmap? It's unclear, but it seems that it doesn't, which I think leads to eventually hitting this assert in update_active_workers(uint v):

    assert(v != 0, "Trying to set active workers to 0");

If that's not the case, please explain how 0 is handled. In any case, I think it should be made consistent with jmap and should also be documented in the above help output:

  parallel=<count> generate histogram using this many parallel threads, default 0
                     0 use system determined number of threads
                     1 use one thread, i.e., disable parallelism
                     n use n threads, n must be positive

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Chris for reviewing, I have updated the patch as you suggested, would you mind to have another look at it?
CSR is created at https://bugs.openjdk.java.net/browse/JDK-8261105

@plummercj
Copy link
Contributor

/label add serviceability

@openjdk openjdk bot added the serviceability serviceability-dev@openjdk.org label Feb 3, 2021
@openjdk
Copy link

openjdk bot commented Feb 3, 2021

@plummercj
The serviceability label was successfully added.

@linzang
Copy link
Contributor

linzang commented Feb 4, 2021

Hi @Hamlin-Li
I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

@plummercj
Copy link
Contributor

I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

That test is for the jmap command, not the GC.class_histogram jcmd. As I pointed out, there are tests in test/hotpot/jtreg/serviceability/dcmd/gc that already test GC.class_histogram. That would be a better starting point.

@linzang
Copy link
Contributor

linzang commented Feb 4, 2021

I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

That test is for the jmap command, not the GC.class_histogram jcmd. As I pointed out, there are tests in test/hotpot/jtreg/serviceability/dcmd/gc that already test GC.class_histogram. That would be a better starting point.

Correct! Thanks for point it out, and sorry for misleading the test case.

@Hamlin-Li
Copy link
Author

Hi @Hamlin-Li
I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

Hi @linzang , it's ok, anyone can review. :-)

@Hamlin-Li
Copy link
Author

@plummercj Hi Chris, I have a question which might not be related to the patch, my presubmit tests always failed on linux platforms, but I don't think the failure is related to my patch. Would you mind to share some point? I totally have no idea what's going wrong.
The error messages are as following:

E: Could not configure 'libc6:i386'.
E: Could not perform immediate configuration on 'libgcc-s1:i386'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)
Error: Process completed with exit code 100.

@plummercj
Copy link
Contributor

From what I've seen it's normal for the 32-bit builds to fail. I'm not sure why they are even being built.

@Hamlin-Li
Copy link
Author

From what I've seen it's normal for the 32-bit builds to fail. I'm not sure why they are even being built.

Thanks for the info, then I think it's safe for me to ignore the failures in linux x86.

@Hamlin-Li
Copy link
Author

/csr

@openjdk openjdk bot added the csr Pull request needs approved CSR before integration label Feb 4, 2021
@openjdk
Copy link

openjdk bot commented Feb 4, 2021

@Hamlin-Li this pull request will not be integrated until the CSR request JDK-8261105 for issue JDK-8261034 has been approved.

@mlbridge
Copy link

mlbridge bot commented Feb 4, 2021

Mailing list message from David Holmes on hotspot-runtime-dev:

On 4/02/2021 5:39 pm, Hamlin Li wrote:

On Thu, 4 Feb 2021 03:40:26 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

Hi @Hamlin-Li
I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

That test is for the `jmap` command, not the `GC.class_histogram` jcmd. As I pointed out, there are tests in `test/hotpot/jtreg/serviceability/dcmd/gc` that already test `GC.class_histogram`. That would be a better starting point.

@plummercj Hi Chris, I have a question which might not be related to the patch, my presubmit tests always failed on linux platforms, but I don't think the failure is related to my patch. Would you mind to share some point? I totally have no idea what's going wrong.
The error messages are as following:

E: Could not configure 'libc6:i386'.
E: Could not perform immediate configuration on 'libgcc-s1:i386'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)
Error: Process completed with exit code 100.

I believe this has been fixed, so your repo may be missing that
changeset at the moment.

David

@Hamlin-Li
Copy link
Author

Mailing list message from David Holmes on hotspot-runtime-dev:

On 4/02/2021 5:39 pm, Hamlin Li wrote:

On Thu, 4 Feb 2021 03:40:26 GMT, Chris Plummer wrote:

Hi @Hamlin-Li
I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

I am not reviewer, but maybe you could add test for "parallel" in BasicJMapTest.java.

That test is for the jmap command, not the GC.class_histogram jcmd. As I pointed out, there are tests in test/hotpot/jtreg/serviceability/dcmd/gc that already test GC.class_histogram. That would be a better starting point.

@plummercj Hi Chris, I have a question which might not be related to the patch, my presubmit tests always failed on linux platforms, but I don't think the failure is related to my patch. Would you mind to share some point? I totally have no idea what's going wrong.
The error messages are as following:
E: Could not configure 'libc6:i386'.
E: Could not perform immediate configuration on 'libgcc-s1:i386'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)
Error: Process completed with exit code 100.

I believe this has been fixed, so your repo may be missing that
changeset at the moment.

David

Hi David, Thank you for the information.

Comment on lines 94 to 96
{"-parallel=0"},
{"-parallel=1"},
{"-parallel=2"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to test invalid arguments within this test framework? It seems the assumption for the run() method is that the arguments are valid and a histogram should be in the output.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, just added some test cases for invalid args, and also added test condition in production to return if parallel < 0.

Comment on lines 559 to 562
"parallel threads number for heap iteration. "
"0 use system determined number of threads, "
"1 use one thread, i.e., disable parallelism, "
"n use n threads, n must be positive.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

   "Number of parallel threads for heap iteration. "
   "0 means let the VM determined the number of threads. "
   "1 means use one thread, i.e. disable parallelism. "
   "n means use n threads. n must be positive.",

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for detailed review, Chris, just modified as you suggested.

"BOOLEAN", false, "false"),
_parallel_thread_num("-parallel",
"Number of parallel threads for heap iteration. "
"0 means let the VM determined the number of threads. "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be "determine". Sorry that was a typo in the text I suggested.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, thanks for pointing out. :-)

Copy link
Contributor

@plummercj plummercj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copyrights need updating in two of the files.

@linzang
Copy link
Contributor

linzang commented Apr 25, 2021

Hi @Hamlin-Li @plummercj, @sspitsyn
I think maybe this PR can be pushed now because in #2261, we finally decided to not expose parallel= option for jmap -dump.
Do you think it is reasonable?

Thanks,
Lin

@Hamlin-Li
Copy link
Author

Thanks Lin for reminding.

@Hamlin-Li
Copy link
Author

Seems the build failures on windows are not related to this change.
So, I think this change is ready for a reviewing.

@Hamlin-Li
Copy link
Author

@plummercj @sspitsyn Kindly reminder. Thanks.

@openjdk openjdk bot removed the csr Pull request needs approved CSR before integration label May 11, 2021
@openjdk
Copy link

openjdk bot commented May 11, 2021

@Hamlin-Li This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8261034: improve jcmd GC.class_histogram to support parallel

Reviewed-by: cjplummer, sspitsyn

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 1646 new commits pushed to the master branch:

  • dfe8833: 8266783: java/lang/reflect/Proxy/DefaultMethods.java fails with jtreg 6
  • 995e956: 8266225: jarsigner is using incorrect security property to show weakness of certs
  • 0a12605: 8265448: (zipfs): Reduce read contention in ZipFileSystem
  • acf02ed: 8208237: Re-examine defmeth tests and update as needed
  • ac0287f: 8266770: Clean pending exception before running dynamic CDS dump
  • 7a0a57c: 8266820: micro java/nio/SelectorWakeup.java has wrong copyright header
  • d0daa72: 8266857: PipedOutputStream.sink should be volatile
  • 381de0c: 8266753: jdk/test/lib/process/ProcTest.java failed with "Exception: Proc abnormal end"
  • 2d2cd78: 8266761: AssertionError in sun.net.httpserver.ServerImpl.responseCompleted
  • 9c9c47e: 8266813: Shenandoah: Use shorter instruction sequence for checking if marking in progress
  • ... and 1636 more: https://git.openjdk.java.net/jdk/compare/35c9da70316da21a6be2fd92a5e5b5f193bdd9d6...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label May 11, 2021
Copy link
Contributor

@sspitsyn sspitsyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks good to me.
Thanks,
Serguei

@Hamlin-Li
Copy link
Author

Thanks @plummercj @sspitsyn for your review.

/integrate

@openjdk openjdk bot closed this May 12, 2021
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels May 12, 2021
@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li Since your change was applied there have been 1655 commits pushed to the master branch:

  • ed32e02: 8241187: ToolBox::grep should allow for negative filtering
  • cc03734: 8266925: Add a test to verify that hidden class's members are not statically invocable
  • 271a0c7: 8047218: [TEST_BUG] java/awt/FullScreen/AltTabCrashTest/AltTabCrashTest.java fails with exception
  • 1a0ff28: 8255035: Update BCEL to Version 6.5.0
  • 57c6ba6: 8266822: Rename MetaspaceShared::is_old_class to be more explicit about what "old" means
  • 616244f: 8266937: Remove Compile::reshape_address
  • 974b9f7: 8266773: Release VM is broken with GCC 9 after 8214237
  • f6c5a6b: 8266784: java/text/Collator/RuleBasedCollatorTest.java fails with jtreg 6
  • 1356116: 8266456: Replace direct TKit.run() calls with jdk.jpackage.test.Annotations.Test annotation
  • dfe8833: 8266783: java/lang/reflect/Proxy/DefaultMethods.java fails with jtreg 6
  • ... and 1645 more: https://git.openjdk.java.net/jdk/compare/35c9da70316da21a6be2fd92a5e5b5f193bdd9d6...master

Your commit was automatically rebased without conflicts.

Pushed as commit 3c47cab.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

10 similar comments
@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@openjdk
Copy link

openjdk bot commented May 12, 2021

@Hamlin-Li The command integrate can only be used in open pull requests.

@Hamlin-Li Hamlin-Li deleted the jcmd-class_histogram branch June 30, 2021 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime hotspot-runtime-dev@openjdk.org integrated Pull request has been integrated serviceability serviceability-dev@openjdk.org
Development

Successfully merging this pull request may close these issues.

4 participants