Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove teardown hook to allow hard shutdown #6395

Merged
merged 1 commit into from Sep 19, 2020

Conversation

headius
Copy link
Member

@headius headius commented Sep 17, 2020

This is a cherry-pick of the crucial commit from #6213 that
should also fix #6379, where users reported that at_exit and
other "clean shutdown" hooks were improperly running during a
"hard shutdown" using exit!.

See c941c41 for the original commit and #6379 (comment) for a discussion of this fix.

This is a cherry-pick of the crucial commit from jruby#6213 that
should also fix jruby#6379, where users reported that at_exit and
other "clean shutdown" hooks were improperly running during a
"hard shutdown" using `exit!`.

The original commit message for c941c41 follows:

Remove JVM shutdown hook for teardown

There's not a lot of context on the commit from 2013 that added
this, but I suspect it's not really correct behavior anymore. In
the process of reordering teardown (jruby#6213) this logic began to
cause a spec hang for Kernel.exit! specs due to the teardown
deadlocking with the main thread:

* Main thread starts up and joins a thread that calls exit!. The
  java.lang.Thread.join call locks the thread object.
* Thread calls exit!, which calls System.exit (as opposed to the
  non-bang exit, which terminates the main thread and lets it tear
  things down safely).
* System exit calls this teardown hook, which attempts to shut
  terminate and join remaining threads.
* Because the teardown is not called from the main thread, the
  system deadlocks trying to shut down the child thread, which is
  locked in the main thread doing join.

The chain of deadlock is main => thread 1 join => system exit =>
thread 2 for teardown => try to kill and join thread 1. The
ultimate deadlock is due to a circular sequence of calls that
can't be interrupted (System.exit's wait on the teardown thread)
combined with the teardown happening from a non-main thread.

It's not clear why this started to hang after the reordering.
Moving the ThreadService teardown back to its former relative
position does not fix the hang, and logically I don't know why it
would... there should still be a deadlock if main is joining a
thread and the System exit hook tries to tear that thread down.
@headius headius added this to the JRuby 9.2.14.0 milestone Sep 17, 2020
@headius headius merged commit 9d64c9c into jruby:jruby-9.2 Sep 19, 2020
1 check passed
@headius headius deleted the no_teardown_hook branch September 19, 2020 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant