8282712: VMConnection.open() does not detect if VM failed to be created, resulting in NPE

[plummercj]

VMConnection.open() expects launchTarget(), attachTarget(), and listentTarget() to either throw an exception or return a valid VirtualMachine instance. Instead they were catching certain exceptions and returning null, which resulted in an NPE in VMConnection.open(). I've fixed it so these APIs now rethrow any caught exception and never return null.

Tested with tier1 and also running the following 10x each on linux-x64, macosx-aarch64, and windows-x64:

test/hotspot/jtreg/vmTestbase/nsk/jdwp/
test/hotspot/jtreg/vmTestbase/nsk/jdb/
test/hotspot/jtreg/vmTestbase/nsk/jdi
test/jdk/com/sun/jdi

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8282712: VMConnection.open() does not detect if VM failed to be created, resulting in NPE (Bug - P5)

Reviewers

• Serguei Spitsyn (@sspitsyn - Reviewer)
• Alex Menkov (@alexmenkov - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15255/head:pull/15255
$ git checkout pull/15255

Update a local copy of the PR:
$ git checkout pull/15255
$ git pull https://git.openjdk.org/jdk.git pull/15255/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15255

View PR using the GUI difftool:
$ git pr show -t 15255

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15255.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back cjplummer! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@plummercj The following label will be automatically applied to this pull request:

• serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (77413305)

[sspitsyn]

Looks okay.
Should the bug have the label "noreg-self"?
Thanks,
Serguei

[openjdk]

@plummercj This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8282712: VMConnection.open() does not detect if VM failed to be created, resulting in NPE

Reviewed-by: sspitsyn, amenkov

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 113 new commits pushed to the master branch:

• f143380: 8314240: test/jdk/sun/security/pkcs/pkcs7/SignerOrder.java fails to compile
• 6b396da: 8062795: (fs) Files.setPermissions requires read access when NOFOLLOW_LINKS specified
• 7b28d36: 8314330: java/foreign tests should respect vm flags when start new processes
• b32d641: 8311943: Cleanup usages of toLowerCase() and toUpperCase() in java.base
• 13f6450: 8313765: Invalid CEN header (invalid zip64 extra data field size)
• 24e896d: 8310275: Bug in assignment operator of ReservedMemoryRegion
• 1925508: 8314144: gc/g1/ihop/TestIHOPStatic.java fails due to extra concurrent mark with -Xcomp
• b80001d: 8314209: Wrong @SInCE tag for RandomGenerator::equiDoubles
• ef6db5c: 8314211: Add NativeLibraryUnload event
• 49ddb19: 8313760: [REDO] Enhance AES performance
• ... and 103 more: https://git.openjdk.org/jdk/compare/87b08b6e0192d88025c2275c7dd2c4bdecda58e8...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[plummercj]

Should the bug have the label "noreg-self"?

There are two fixes of almost identical code. One is in test infra code, and thus maybe a noreg-self is warranted, but the other is in jdb, so I'm not sure what is best in that case.

[sspitsyn]

Should the bug have the label "noreg-self"?

There are two fixes of almost identical code. One is in test infra code, and thus maybe a noreg-self is warranted, but the other is in jdb, so I'm not sure what is best in that case.

You are right. Part of this is in jdb, so no label is needed.

[plummercj]

Ping! I could use one more review please. Thanks

[plummercj]

Thank you Alex and Serguei!

/integrate

[openjdk]

Going to push as commit 388dcff.
Since your change was applied there have been 124 commits pushed to the master branch:

• e8f6b3e: 8314268: Missing include in assembler_riscv.hpp
• c634bdf: 8314444: Update jib-profiles.js to use JMH 1.37 devkit
• 2b81885: 8314071: Test java/foreign/TestByteBuffer.java timed out
• 32efd23: 8311939: Excessive allocation of Matcher.groups array
• ed585d1: 8314280: StructuredTaskScope.shutdown should document that the state of completing subtasks is not defined
• 6f1071f: 8314213: DocLint should warn about unknown standard tags
• 4331193: 8314423: Multiple patterns without unnamed variables
• 249dc37: 8314321: Remove unused field jdk.internal.util.xml.impl.Attrs.mAttrIdx
• b78f5a1: 8314076: ICC_ColorSpace#minVal/maxVal have the opposite description
• 2a1176b: 8314276: Improve PtrQueue API around size/capacity
• ... and 114 more: https://git.openjdk.org/jdk/compare/87b08b6e0192d88025c2275c7dd2c4bdecda58e8...master

Your commit was automatically rebased without conflicts.

[openjdk]

@plummercj Pushed as commit 388dcff.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.