8254877: GCLogPrecious::_lock rank constrains what locks you are allowed to have when crashing #903

stefank · 2020-10-28T13:49:15Z

This is an alternative version of the fix proposed in 900:
#900

Erik's description:

Today, when you crash, the GCLogPrecious::_lock is taken. This effectively limits you to only get clean crash reports if you crash or assert without holding a lock of rank tty or lower. It is arguably difficult to know what locks you are going to have when crashing. Therefore, I don't think the precious GC log should constrain possible crashing contexts in that fashion.

As Erik mentioned in that PR, I'd like to retain the ability to easily dump the precious log when debugging. The proposed fix changes the Mutex to a Semaphore, and use trywait to safely access the buffer. In the unlikely event that another thread is holding the lock, the hs_err printer skips printing the log.

This also makes it possible to call precious logging from within the stack watermark processing code. I think there's a possibility that we might call the following error logging, when we fail to commit memory for a ZPage, when relocating, during stack watermark processing:
log_error_p(gc)("Failed to commit memory (%s)", err.to_string());

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Testing

	Linux additional	Linux x64	Linux x86	Windows x64	macOS x64
Build	✔️ (8/8 passed)	✔️ (2/2 passed)	✔️ (2/2 passed)	✔️ (2/2 passed)	✔️ (2/2 passed)
Test (tier1)		✔️ (9/9 passed)	❌ (2/9 failed)	✔️ (9/9 passed)	✔️ (9/9 passed)

Failed test tasks

Issue

JDK-8254877: GCLogPrecious::_lock rank constrains what locks you are allowed to have when crashing

Reviewers

Erik Österlund (@fisk - Reviewer) ⚠️ Review applies to 9888692

Download

$ git fetch https://git.openjdk.java.net/jdk pull/903/head:pull/903
$ git checkout pull/903

…wed to have when crashing

bridgekeeper · 2020-10-28T13:49:42Z

👋 Welcome back stefank! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-10-28T13:50:50Z

@stefank The following label will be automatically applied to this pull request:

hotspot-gc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2020-10-28T13:53:50Z

Webrevs

pliden · 2020-10-28T15:02:40Z

src/hotspot/share/gc/shared/gcLogPrecious.cpp

+    st->print_cr("%s", _lines->base());
+  }
+
+  _lock->signal();


As we discussed off-line, perhaps we might want to print something even if the log isn't initialized and/or is empty. Something like:

st->print_cr("GC Precious Log:"); if (_lines == NULL) { st->print_cr("<Not initialized>"); return; } if (!_lock->trywait()) { st->print_cr("<Skipped>"); return; } if (_lines->size() == 0) { st->print_cr("<Empty>"); } else { st->print_cr("%s", _lines->base()); } _lock->signal();

You decide. Looks good otherwise.

Updated with suggestion. I also added extra newlines to make the output look pretty. That was needed because _lines->base() is always terminated with a newline.

openjdk · 2020-10-28T15:04:44Z

@stefank This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254877: GCLogPrecious::_lock rank constrains what locks you are allowed to have when crashing

Reviewed-by: eosterlund

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 6 new commits pushed to the master branch:

bf66d73: 8257073: ZGC: Try forward object before retaining page
1b3aa3a: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8
734d3c3: 8256862: Several java/foreign tests fail on x86_32 platforms
7946c94: 8257082: ZGC: Clean up ZRuntimeWorkers and ZWorkers
f6d6a07: 8256938: Improve remembered set sampling task scheduling
b823ad9: 8257072: ZGC: Rename roots iterators

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

fisk

Looks good.

albertnetymk · 2020-10-28T16:02:37Z

src/hotspot/share/gc/shared/gcLogPrecious.cpp


 stringStream* GCLogPrecious::_lines = NULL;
 stringStream* GCLogPrecious::_temp = NULL;
-Mutex* GCLogPrecious::_lock = NULL;
+Semaphore* GCLogPrecious::_lock = NULL;


Maybe renaming _lock to _semaphore? Additionally, since it's a binary semaphore, new Semaphore(1), some comments explaining why Mutex is not suitable could avoid some future confusions.

PS: not a review, just a comment in passing.

It's used as a lock, so I think the name _lock is appropriate. Instead I introduced a new class: SemaphoreLock, to make the code more readable (IMHO). Also added a comment. Hopefully, this addressed your comments.

Thank you; it does look more readable. BTW, mutex has try_lock as well.

I see. Then I'll remove that part of the new comment.

stefank · 2020-10-29T09:35:45Z

In the latest update I added two new helper classes: SemaphoreLock and SemaphoreLocker. I think this makes the code nicer. Since those classes are more broadly used, I'll go a head and split them out into a separate PR.

stefank · 2020-10-29T10:03:30Z

Forked off the SemaphoreLock part into #927

pliden · 2020-10-29T11:07:31Z

Updates look good!

…wed to have when crashing

openjdk · 2020-11-24T12:09:38Z

⚠️ @stefank This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

…precious_locks

stefank · 2020-11-24T12:16:07Z

I've thrown away the Semaphore implementation and replaced it with Patricio's new try_lock_without_range_check function. I've also changed the lock rank to be as low as possible. We'll still get a lock rank reordering problem if we crash while holding this lock, because EventLog takes its lock and it is of the same rank. I intend to address that with JDK-8256382.

pliden

Update looks good.

fisk

Looks good. Quite entertaining that the "leaf" rank is so far away from being a leaf level rank nowadays.

stefank · 2020-11-24T14:57:24Z

Looks good. Quite entertaining that the "leaf" rank is so far away from being a leaf level rank nowadays.

I had the exact same thought. :(

Thanks for reviewing @pliden and @fisk

stefank · 2020-12-02T12:27:53Z

/integrate

openjdk · 2020-12-02T12:29:03Z

@stefank Since your change was applied there have been 97 commits pushed to the master branch:

1fd0ea7: 8256382: Use try_lock for hs_err EventLog printing
bff68f1: 8257533: legacy-jre-image includes jpackage and jlink tools
9a60413: 8248736: [aarch64] runtime/signal/TestSigpoll.java failed "fatal error: not an ldr (literal) instruction."
e7ca0c4: 8257224: JDK-8251549 didn't update building.html
7e37c7c: 8257471: fatal error: Fatal exception in JVMCI: Exception during JVMCI compiler initialization
3e3745c: 8256008: UL does not report anything if disk writing fails
fb139cf: 8257467: [TESTBUG] -Wdeprecated-declarations is reported at sigset() in exesigtest.c
9de283b: 8257505: nsk/share/test/StressOptions stressTime is scaled in getter but not when printed
282cb32: 8005970: Mouse cursor is default cursor over TextArea's scrollbar
f2a0988: 8257228: G1: SIGFPE in G1ConcurrentRefine::create(int*) due to buffers_to_cards overflow
... and 87 more: https://git.openjdk.java.net/jdk/compare/973255c469d794afe8ee74b24ddb5048bfcaadf7...master

Your commit was automatically rebased without conflicts.

Pushed as commit 287b829.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8254877: GCLogPrecious::_lock rank constrains what locks you are allo…

fdadc38

…wed to have when crashing

openjdk bot added the rfr Pull request is ready for review label Oct 28, 2020

openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label Oct 28, 2020

pliden approved these changes Oct 28, 2020

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 28, 2020

fisk approved these changes Oct 28, 2020

View reviewed changes

albertnetymk reviewed Oct 28, 2020

View reviewed changes

Review 1

88e2b4e

openjdk bot added hotspot hotspot-dev@openjdk.org and removed hotspot-gc hotspot-gc-dev@openjdk.org labels Oct 29, 2020

Review 2

63a9473

stefank mentioned this pull request Oct 29, 2020

8255582: Introduce SemaphoreLock and SemaphoreLocker #927

Closed

3 tasks

8254877: GCLogPrecious::_lock rank constrains what locks you are allo…

3df441f

…wed to have when crashing

Merge branch '8254877_gclogprecious_locks_rebased' into 8254877_gclog…

9888692

…precious_locks

stefank force-pushed the 8254877_gclogprecious_locks branch from fa17f12 to 9888692 Compare November 24, 2020 12:10

openjdk bot added hotspot-gc hotspot-gc-dev@openjdk.org and removed hotspot hotspot-dev@openjdk.org labels Nov 24, 2020

pliden reviewed Nov 24, 2020

View reviewed changes

fisk approved these changes Nov 24, 2020

View reviewed changes

Merge branch 'master' into 8254877_gclogprecious_locks

be0da01

openjdk bot closed this Dec 2, 2020

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Dec 2, 2020

stefank deleted the 8254877_gclogprecious_locks branch January 13, 2021 09:10

8254877: GCLogPrecious::_lock rank constrains what locks you are allowed to have when crashing #903

8254877: GCLogPrecious::_lock rank constrains what locks you are allowed to have when crashing #903

stefank commented Oct 28, 2020 •

edited by openjdk bot

bridgekeeper bot commented Oct 28, 2020

openjdk bot commented Oct 28, 2020

mlbridge bot commented Oct 28, 2020 •

edited

pliden Oct 28, 2020

stefank Oct 29, 2020

openjdk bot commented Oct 28, 2020 •

edited

fisk left a comment

albertnetymk Oct 28, 2020

stefank Oct 29, 2020

albertnetymk Oct 29, 2020

stefank Oct 29, 2020

stefank commented Oct 29, 2020

stefank commented Oct 29, 2020

pliden commented Oct 29, 2020

openjdk bot commented Nov 24, 2020

stefank commented Nov 24, 2020

pliden left a comment •

edited

fisk left a comment

stefank commented Nov 24, 2020

stefank commented Dec 2, 2020

openjdk bot commented Dec 2, 2020

8254877: GCLogPrecious::_lock rank constrains what locks you are allowed to have when crashing #903

8254877: GCLogPrecious::_lock rank constrains what locks you are allowed to have when crashing #903

Conversation

stefank commented Oct 28, 2020 • edited by openjdk bot

Progress

Testing

Issue

Reviewers

Download

bridgekeeper bot commented Oct 28, 2020

openjdk bot commented Oct 28, 2020

mlbridge bot commented Oct 28, 2020 • edited

Webrevs

pliden Oct 28, 2020

Choose a reason for hiding this comment

stefank Oct 29, 2020

Choose a reason for hiding this comment

openjdk bot commented Oct 28, 2020 • edited

fisk left a comment

Choose a reason for hiding this comment

albertnetymk Oct 28, 2020

Choose a reason for hiding this comment

stefank Oct 29, 2020

Choose a reason for hiding this comment

albertnetymk Oct 29, 2020

Choose a reason for hiding this comment

stefank Oct 29, 2020

Choose a reason for hiding this comment

stefank commented Oct 29, 2020

stefank commented Oct 29, 2020

pliden commented Oct 29, 2020

openjdk bot commented Nov 24, 2020

stefank commented Nov 24, 2020

pliden left a comment • edited

Choose a reason for hiding this comment

fisk left a comment

Choose a reason for hiding this comment

stefank commented Nov 24, 2020

stefank commented Dec 2, 2020

openjdk bot commented Dec 2, 2020

stefank commented Oct 28, 2020 •

edited by openjdk bot

mlbridge bot commented Oct 28, 2020 •

edited

openjdk bot commented Oct 28, 2020 •

edited

pliden left a comment •

edited