8267517: async logging for stdout and stderr #1555

navyxliu · 2023-07-07T00:27:09Z

This PR backports JDK-8267517 to jdk17u. -Xlog:async will route LogOutput stdout/stderr to the asynchronous logging queue like ordinary log files. The reason we do is that some Java application outputs logs to stdout/stderr and redirect them to files then.

This is NOT a clean backport. It's because jdk17u is missing a few patches, such as JDK-8273471, JDK-8269004. I tried to backport them beforehand, but they both have dependencies. I manually resolve a couple of conflicts.

We have verified patch like how we verified for jdk-tip. We output verbose gclog to stdout -Xlog:gc=trace:stdout and leverage software flow control to pause our console(C-s). Java threads don't be blocked with '-Xlog:async'. Without this patch, kernel puts Java threads which require GC to 'sleep' state when unified log is blocked.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Error

⚠️ The pull request body must not be empty.

Issue

JDK-8267517: async logging for stdout and stderr (Enhancement - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk17u-dev.git pull/1555/head:pull/1555
$ git checkout pull/1555

Update a local copy of the PR:
$ git checkout pull/1555
$ git pull https://git.openjdk.org/jdk17u-dev.git pull/1555/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 1555

View PR using the GUI difftool:
$ git pr show -t 1555

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk17u-dev/pull/1555.diff

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8267517: async logging for stdout and stderr (Enhancement - P4)

Reviewers

Aleksey Shipilev (@shipilev - Reviewer) ⚠️ Review applies to e34247cb

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk17u-dev.git pull/1555/head:pull/1555
$ git checkout pull/1555

Update a local copy of the PR:
$ git checkout pull/1555
$ git pull https://git.openjdk.org/jdk17u-dev.git pull/1555/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 1555

View PR using the GUI difftool:
$ git pr show -t 1555

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk17u-dev/pull/1555.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2023-07-07T00:28:20Z

👋 Welcome back xliu! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-07-07T00:30:08Z

This backport pull request has now been updated with issue from the original commit.

mlbridge · 2023-07-07T05:35:08Z

Webrevs

01: Full - Incremental (855a095d)
00: Full (e34247cb)

shipilev

Looks okay with minor nit.

src/hotspot/share/logging/logFileStreamOutput.cpp

shipilev · 2023-07-10T09:38:09Z

Please also run tier1 tier2 tier3 on any convenient architecture for this.

openjdk · 2023-07-10T09:39:31Z

@navyxliu This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8267517: async logging for stdout and stderr

Reviewed-by: shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 25 new commits pushed to the master branch:

9288072: 8292716: Configure should check that jtreg is of the required version
1248b15: 8303154: Investigate and improve instruction cache flushing during compilation
4eeb1c9: 8180266: Convert sun/security/provider/KeyStore/DKSTest.sh to Java Jtreg Test
f45578e: 8225012: sanity/client/SwingSet/src/ToolTipDemoTest.java fails on Windows
a299757: 8297476: Increase InlineSmallCode default from 1000 to 2500 for RISC-V
d5eec58: 8283929: GHA: Add RISC-V build config
f956352: 8305421: Work around JDK-8305420 in CDSJDITest.java
c9194e7: 8306566: Open source several clipboard AWT tests
afffb37: 8306076: Open source AWT misc tests
b42886f: 8306636: Disable compiler/c2/Test6905845.java with -XX:TieredStopAtLevel=3
... and 15 more: https://git.openjdk.org/jdk17u-dev/compare/fc08b730142dcde39291be8044dfa667b98363ad...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@shipilev) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

navyxliu · 2023-07-11T08:34:29Z

Please also run tier1 tier2 tier3 on any convenient architecture for this.

hi, Aleksey,

Thank you for reviewing this change.

Github workflow ran tier1 tests for us. I ran tier2/3 on my x86_64 instance. Those 2 failures are not caused by this change but my platform. tier1/2/3 are clear.

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
>> jtreg:test/hotspot/jtreg:tier2                      701   699     2     0 <<
>> jtreg:test/jdk:tier2                               3814  3813     1     0 <<
   jtreg:test/langtools:tier2                           11    11     0     0
   jtreg:test/jaxp:tier2                               465   465     0     0
   jtreg:test/hotspot/jtreg:tier3                      212   212     0     0
   jtreg:test/jdk:tier3                               1301  1301     0     0
   jtreg:test/langtools:tier3                            0     0     0     0
   jtreg:test/jaxp:tier3                                 0     0     0     0
==============================

GoeLin · 2023-07-12T06:16:13Z

Thanks for this thoroughly documented backport!

navyxliu · 2023-07-12T17:24:23Z

/integrate

openjdk · 2023-07-12T17:26:08Z

@navyxliu
Your change (at version 855a095) is now ready to be sponsored by a Committer.

shipilev · 2023-07-12T17:30:02Z

/sponsor

openjdk · 2023-07-12T17:32:24Z

Going to push as commit e61bb2e.
Since your change was applied there have been 25 commits pushed to the master branch:

9288072: 8292716: Configure should check that jtreg is of the required version
1248b15: 8303154: Investigate and improve instruction cache flushing during compilation
4eeb1c9: 8180266: Convert sun/security/provider/KeyStore/DKSTest.sh to Java Jtreg Test
f45578e: 8225012: sanity/client/SwingSet/src/ToolTipDemoTest.java fails on Windows
a299757: 8297476: Increase InlineSmallCode default from 1000 to 2500 for RISC-V
d5eec58: 8283929: GHA: Add RISC-V build config
f956352: 8305421: Work around JDK-8305420 in CDSJDITest.java
c9194e7: 8306566: Open source several clipboard AWT tests
afffb37: 8306076: Open source AWT misc tests
b42886f: 8306636: Disable compiler/c2/Test6905845.java with -XX:TieredStopAtLevel=3
... and 15 more: https://git.openjdk.org/jdk17u-dev/compare/fc08b730142dcde39291be8044dfa667b98363ad...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-07-12T17:32:37Z

@shipilev @navyxliu Pushed as commit e61bb2e.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Backport 7aff03aee1a5cd70638d8bbafeb7765bc7beea45

e34247c

openjdk bot changed the title ~~Backport 7aff03aee1a5cd70638d8bbafeb7765bc7beea45~~ 8267517: async logging for stdout and stderr Jul 7, 2023

openjdk bot added backport Port of a pull request already in a different code base rfr Pull request is ready for review labels Jul 7, 2023

shipilev approved these changes Jul 10, 2023

View reviewed changes

src/hotspot/share/logging/logFileStreamOutput.cpp Outdated Show resolved Hide resolved

openjdk bot added the ready Pull request is ready to be integrated label Jul 10, 2023

update per feedback

855a095

openjdk bot added the sponsor Pull request is ready to be sponsored label Jul 12, 2023

openjdk bot added the integrated Pull request has been integrated label Jul 12, 2023

openjdk bot closed this Jul 12, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Jul 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

8267517: async logging for stdout and stderr #1555

8267517: async logging for stdout and stderr #1555

Uh oh!

navyxliu commented Jul 7, 2023 •

edited by openjdk bot

Loading

Uh oh!

bridgekeeper bot commented Jul 7, 2023

Uh oh!

openjdk bot commented Jul 7, 2023

Uh oh!

mlbridge bot commented Jul 7, 2023 •

edited

Loading

Uh oh!

shipilev left a comment

Uh oh!

Uh oh!

shipilev commented Jul 10, 2023

Uh oh!

openjdk bot commented Jul 10, 2023 •

edited

Loading

Uh oh!

navyxliu commented Jul 11, 2023

Uh oh!

GoeLin commented Jul 12, 2023

Uh oh!

navyxliu commented Jul 12, 2023

Uh oh!

openjdk bot commented Jul 12, 2023

Uh oh!

shipilev commented Jul 12, 2023

Uh oh!

openjdk bot commented Jul 12, 2023

Uh oh!

openjdk bot commented Jul 12, 2023

Uh oh!

Uh oh!

8267517: async logging for stdout and stderr #1555

8267517: async logging for stdout and stderr #1555

Uh oh!

Conversation

navyxliu commented Jul 7, 2023 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Error

Issue

Reviewing

Progress

Issue

Reviewers

Reviewing

Webrev

Uh oh!

bridgekeeper bot commented Jul 7, 2023

Uh oh!

openjdk bot commented Jul 7, 2023

Uh oh!

mlbridge bot commented Jul 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

shipilev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shipilev commented Jul 10, 2023

Uh oh!

openjdk bot commented Jul 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

navyxliu commented Jul 11, 2023

Uh oh!

GoeLin commented Jul 12, 2023

Uh oh!

navyxliu commented Jul 12, 2023

Uh oh!

openjdk bot commented Jul 12, 2023

Uh oh!

shipilev commented Jul 12, 2023

Uh oh!

openjdk bot commented Jul 12, 2023

Uh oh!

openjdk bot commented Jul 12, 2023

Uh oh!

Uh oh!

navyxliu commented Jul 7, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented Jul 7, 2023 •

edited

Loading

openjdk bot commented Jul 10, 2023 •

edited

Loading