Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8263567: gtests don't terminate the VM safely #4986

Closed
wants to merge 4 commits into from

Conversation

dholmes-ora
Copy link
Member

@dholmes-ora dholmes-ora commented Aug 4, 2021

Please review this fix to shutdown cleanly any JVMs created by gtests that aren't aborted.

Testing:

  • manual instrumentation to check all JVM construction and then deletion during gtest runs
  • local gtest testing
  • tiers 1-3 gtest testing

Thanks,
David


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4986/head:pull/4986
$ git checkout pull/4986

Update a local copy of the PR:
$ git checkout pull/4986
$ git pull https://git.openjdk.java.net/jdk pull/4986/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4986

View PR using the GUI difftool:
$ git pr show -t 4986

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4986.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Aug 4, 2021

👋 Welcome back dholmes! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Aug 4, 2021

@dholmes-ora The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot-runtime label Aug 4, 2021
@dholmes-ora
Copy link
Member Author

@dholmes-ora dholmes-ora commented Aug 4, 2021

/label add hotspot

@openjdk openjdk bot added the hotspot label Aug 4, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 4, 2021

@dholmes-ora
The hotspot label was successfully added.

@dholmes-ora dholmes-ora marked this pull request as ready for review Aug 4, 2021
@openjdk openjdk bot added the rfr label Aug 4, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Aug 4, 2021

Webrevs

Copy link
Member

@tstuefe tstuefe left a comment

Hi David,

looks good, small remarks inline and one general remark here:

I wonder whether we should not just always initialize the VM at launch and destroy it after finish, regardless of the tests we run. We even could abolish TEST and just always do TEST_VM.

While in theory TEST (without _VM) is a good idea because we save time on the VM initialization, in practice

  1. usually one just runs always the full suite, or a subset of it which includes TEST_VM tests, so most of the time we pay for VM init anyway.
  2. tests are done using copy+paste and use TEST_VM unnecessarily. OTOH I also already had TEST tests that needed VM stuff. I mean that maintaining the correct designation for the tests is cumbersome and error-prone.
  3. The _VM flexibility, tied to order and selection of tests, means the time VM initialization happens is unpredictable, which makes analysis more difficult and behavior sometimes surprising.

We could simplify all this and the runner code by just unconditionally initializing the VM at start of the gtestlauncher. I think the benefits of differentiating between TEST and TEST_VM are slim in practice.

..Thomas

test/hotspot/gtest/gtestMain.cpp Outdated Show resolved Hide resolved
test/hotspot/gtest/gtestMain.cpp Outdated Show resolved Hide resolved
test/hotspot/gtest/gtestMain.cpp Outdated Show resolved Hide resolved
if (ret != 0) { \
fprintf(stderr, "Warning: DestroyJavaVM error %d\n", ret); \
} \
} \
Copy link
Member

@tstuefe tstuefe Aug 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For better readibility I'd factor this out to an own function.

Copy link
Member Author

@dholmes-ora dholmes-ora Aug 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is I don't have anywhere to put that function so that this macro can invoke it.

Copy link
Member Author

@dholmes-ora dholmes-ora Aug 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be more clear this is a .hpp file included by many .cpp files. If I put the function in the .hpp file and a specific .cpp file does not use the TEST_OTHER_VM macro then we have a function defined but not used and so hit -Werror=unused-function.
Is there some .cpp file I can put this in that would be globally accessible to all the test .cpp files?

@dholmes-ora
Copy link
Member Author

@dholmes-ora dholmes-ora commented Aug 6, 2021

Hi Thomas,

On 4/08/2021 5:22 pm, Thomas Stuefe wrote:

On Wed, 4 Aug 2021 05:55:11 GMT, David Holmes dholmes@openjdk.org wrote:

Please review this fix to shutdown cleanly any JVMs created by gtests that aren't aborted.

Testing:

  • manual instrumentation to check all JVM construction and then deletion during gtest runs
  • local gtest testing
  • tiers 1-3 gtest testing

Thanks,
David

Hi David,

looks good, small remarks inline and one general remark here:
Thanks for taking a look at this. Sorry for the delayed follow-up.
I wonder whether we should not just always initialize the VM at launch and destroy it after finish, regardless of the tests we run. We even could abolish TEST and just always do TEST_VM.

While in theory TEST (without _VM) is a good idea because we save time on the VM initialization, in practice

  1. usually one just runs always the full suite, or a subset of it which includes TEST_VM tests, so most of the time we pay for VM init anyway.
  2. tests are done using copy+paste and use TEST_VM unnecessarily. OTOH I also already had TEST tests that needed VM stuff. I mean that maintaining the correct designation for the tests is cumbersome and error-prone.
  3. The _VM flexibility, tied to order and selection of tests, means the time VM initialization happens is unpredictable, which makes analysis more difficult and behavior sometimes surprising.

We could simplify all this and the runner code by just unconditionally initializing the VM at start of the gtestlauncher. I think the benefits of differentiating between TEST and TEST_VM are slim in practice.

That may be true but I don't think it is for this issue/PR to try and address that. I can imagine that some basic TEST() cases assume a pristine environment with no chance of any external perturbation (such as by an initialized and thus operating VM). That said with the way tests get interleaved depending on exactly how you run them, it isn't clear that a TEST() may not run when the VM is already initialized anyway!

Thanks,
David

tstuefe
tstuefe approved these changes Aug 6, 2021
Copy link
Member

@tstuefe tstuefe left a comment

All good now. The fact that other_vm re-runs VM initialization tripped me off more than once btw :)

test/hotspot/gtest/gtestMain.cpp Outdated Show resolved Hide resolved
@openjdk
Copy link

@openjdk openjdk bot commented Aug 6, 2021

@dholmes-ora This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8263567: gtests don't terminate the VM safely

Reviewed-by: stuefe, dcubed

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 109 new commits pushed to the master branch:

  • 0ac2be9: 8272123: Problem list 4 jtreg tests which regularly fail on macos-aarch64
  • 272fcb4: 8272113: Build compare script fails with differences in classlist
  • 2f7a469: 8271931: Make AbortVMOnVMOperationTimeout more resilient to OS scheduling
  • a86ac0d: 8271939: Clean up primitive raw accessors in oopDesc
  • b84a9c7: 8271956: AArch64: C1 build failed after JDK-8270947
  • 38ff85c: 8271461: CompileCommand support for hidden class methods
  • c495ede: 8272099: mark hotspot runtime/Monitor tests which ignore external VM flags
  • e882087: 8271904: mark hotspot runtime/ClassFile tests which ignore external VM flags
  • fa36e33: 8271513: support JavaThreadIteratorWithHandle replacement by new ThreadsList::Iterator
  • cc61520: 8270842: G1: Only young regions need to redirty outside references in remset.
  • ... and 99 more: https://git.openjdk.java.net/jdk/compare/4f42eb6601c3b6011d3c2b30af6b2be264ff7c0e...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Aug 6, 2021
@dholmes-ora
Copy link
Member Author

@dholmes-ora dholmes-ora commented Aug 9, 2021

Could I get a second review please.

Thanks.

Copy link
Member

@dcubed-ojdk dcubed-ojdk left a comment

Thumbs up.

@dholmes-ora
Copy link
Member Author

@dholmes-ora dholmes-ora commented Aug 9, 2021

Thanks @dcubed-ojdk !

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Aug 9, 2021

Going to push as commit 843943c.
Since your change was applied there have been 114 commits pushed to the master branch:

  • 7fc99cf: 8225488: Examine ExecutableType.getReceiverType behavior when source receiver parameter is absent
  • 4548677: 8268824: Remove unused jdk.accessibility APIs deprecated for removal in JDK 9
  • b53828b: 8272047: java/nio/channels/FileChannel/Transfer2GPlus.java failed with Unexpected transfer size: 2147418112
  • 41dc795: 8264792: The NumberFormat for locale sq_XK formats price incorrectly.
  • 9c6457f: 8267385: Create NSAccessibilityElement implementation for JavaComponentAccessibility
  • 0ac2be9: 8272123: Problem list 4 jtreg tests which regularly fail on macos-aarch64
  • 272fcb4: 8272113: Build compare script fails with differences in classlist
  • 2f7a469: 8271931: Make AbortVMOnVMOperationTimeout more resilient to OS scheduling
  • a86ac0d: 8271939: Clean up primitive raw accessors in oopDesc
  • b84a9c7: 8271956: AArch64: C1 build failed after JDK-8270947
  • ... and 104 more: https://git.openjdk.java.net/jdk/compare/4f42eb6601c3b6011d3c2b30af6b2be264ff7c0e...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Aug 9, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Aug 9, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 9, 2021

@dholmes-ora Pushed as commit 843943c.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@dholmes-ora dholmes-ora deleted the 8263567-gtest branch Aug 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot hotspot-runtime integrated
3 participants