8343344: Windows attach logic failed to handle a failed open on a pipe

[alexmenkov]

The fix adds correct handling of "could not open pipe" errors in attach listener instead of assert which causes target VM crash.

testing: tier1,tier2,hs-tier5-svc

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8343344: Windows attach logic failed to handle a failed open on a pipe (Bug - P4)

Reviewers

• Kevin Walls (@kevinjwalls - Reviewer)
• Chris Plummer (@plummercj - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/21888/head:pull/21888
$ git checkout pull/21888

Update a local copy of the PR:
$ git checkout pull/21888
$ git pull https://git.openjdk.org/jdk.git pull/21888/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 21888

View PR using the GUI difftool:
$ git pr show -t 21888

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/21888.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back amenkov! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@alexmenkov This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8343344: Windows attach logic failed to handle a failed open on a pipe

Reviewed-by: kevinw, cjplummer

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 36 new commits pushed to the master branch:

• c0e6c3b: 8343214: Fix encoding errors in APX New Data Destination Instructions Support
• 0be7118: 8279016: JFR Leak Profiler is broken with Shenandoah
• 6811a11: 8341408: Implement JEP 488: Primitive Types in Patterns, instanceof, and switch (Second Preview)
• 72a45dd: 8341834: C2 compilation fails with "bad AD file" due to Replicate
• 57c3bb6: 8343068: C2: CastX2P Ideal transformation not always applied
• 83f3d42: 8339303: C2: dead node after failing to match cloned address expression
• ead0116: 8331341: secondary_super_cache does not scale well: C1 and interpreter
• 06d8216: 8318442: java/net/httpclient/ManyRequests2.java fails intermittently on Linux
• bdd6816: 8343502: RISC-V: SIGBUS in updateBytesCRC32 after JDK-8339738
• 4431852: 8342943: Replace predicate walking and cloning code for main/post loops with a predicate visitor
• ... and 26 more: https://git.openjdk.org/jdk/compare/825ceb16b2e2347a4d9c1977d9a3a2da1296d5fe...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[openjdk]

@alexmenkov The following label will be automatically applied to this pull request:

• hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (ec14cd51)
• 00: Full (e535bfcd)

[alexmenkov]

/label add serviceability

[openjdk]

@alexmenkov
The serviceability label was successfully added.

[kevinjwalls]

Hi,

List of tests doesn't mention tier4 where the failures were first noticed?

When PipeChannel::open logs error, and returns false, then Win32AttachOperation::open_pipe returns that value, then Win32AttachListener::dequeue() also logs that there was an error?
Is that message just a less detailed duplicate?
(it states whether it's v1 or v2, but maybe that's not that interesting?)

In the v2 case, Win32AttachListener::dequeue() will log if open succeeded, if !op->read_request().
But AttachOperation::read_request mostly logs its own errors as well.
I think it's only this one:
639 if (buffer_size < 0) {
640 return false;
...where it does not log when returning false. Maybe it should log then, and Win32AttachListener::dequeue() does not need to log?

While are are here, is it possible to address these at the same time?
open/src/hotspot/share/services/attachListener.cpp

typo "ot" -> "or"
631 case ATTACH_API_V2: // 000000
632 if (AttachListener::get_supported_version() < 2) {
633 log_error(attach)("Failed to read request: v2 is unsupported ot disabled");

exact -> "exactly"
649 // Must contain exact 'buffer_size' bytes.

Also, is the line 631 comment, with 3 args, misleading? (moving away from exactly 3 arguments...)

Thanks!

[alexmenkov]

List of tests doesn't mention tier4 where the failures were first noticed?

Good point. Run it now, but it takes time to complete.

When PipeChannel::open logs error, and returns false, then Win32AttachOperation::open_pipe returns that value, then Win32AttachListener::dequeue() also logs that there was an error? Is that message just a less detailed duplicate? (it states whether it's v1 or v2, but maybe that's not that interesting?)
In the v2 case, Win32AttachListener::dequeue() will log if open succeeded, if !op->read_request(). But AttachOperation::read_request mostly logs its own errors as well. I think it's only this one: 639 if (buffer_size < 0) { 640 return false; ...where it does not log when returning false. Maybe it should log then, and Win32AttachListener::dequeue() does not need to log?

Ok, done.
Also added error logging in PipeChannel:: read and write (AttachOperation::read_request_data does not logs IO errors)

While are are here, is it possible to address these at the same time? open/src/hotspot/share/services/attachListener.cpp
typo "ot" -> "or" 631 case ATTACH_API_V2: // 000000 632 if (AttachListener::get_supported_version() < 2) { 633 log_error(attach)("Failed to read request: v2 is unsupported ot disabled");

Fixed.

exact -> "exactly" 649 // Must contain exact 'buffer_size' bytes.

Fixed.

Also, is the line 631 comment, with 3 args, misleading? (moving away from exactly 3 arguments...)

Fixed.

[kevinjwalls]

Thanks Alex, looks good.

[plummercj]

Looks good.

[alexmenkov]

/integrate

[openjdk]

Going to push as commit a4e2c20.
Since your change was applied there have been 127 commits pushed to the master branch:

• 63eb485: 8343883: Cannot resolve user specified toolchain-path for lld after JDK-8338304
• db85090: 8338411: Implement JEP 486: Permanently Disable the Security Manager
• c12b386: 8338007: [JVMCI] ResolvedJavaMethod.reprofile can crash ciMethodData
• 81752c4: 8338565: Test crashed: assert(is_path_empty()) failed: invariant
• e5eaa7f: 8343946: JFR: Wildcard should only work with COUNT for 'jfr view'
• 2989d87: 8343805: RISC-V: JVM crashes on startup when disabling compressed instructions
• 78b8015: 8338383: Implement JEP 491: Synchronize Virtual Threads without Pinning
• 8a2a75e: 8339892: Several security shell tests don't set TESTJAVAOPTS
• 50b6e41: 8300732: Whitebox functions for Metaspace test should use byte size
• 67d1ef1: 8344018: Remove unlimited memory setting from TestScalarReplacementMaxLiveNodes
• ... and 117 more: https://git.openjdk.org/jdk/compare/825ceb16b2e2347a4d9c1977d9a3a2da1296d5fe...master

Your commit was automatically rebased without conflicts.

[openjdk]

@alexmenkov Pushed as commit a4e2c20.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.