-
Notifications
You must be signed in to change notification settings - Fork 478
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid deadlock in ConnectionManager::Stop #2950
Avoid deadlock in ConnectionManager::Stop #2950
Conversation
Signed-off-by: Ivan Santiago Paunovic <ivanpauno@ekumenlabs.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've tested this together with #2949 and it haven't seen any lingering gzserver process after shutdown.
@jacobperron if you have write permissions, please go ahead and merge. (the PR checkers failures doesn't seem related to me, but IDK) |
I was having trouble reproducing this reliably, but with the following patch it happen every time (lol)
|
I merged this branch with |
Signed-off-by: Ivan Santiago Paunovic <ivanpauno@ekumenlabs.com> Co-authored-by: Steve Peters <scpeters@openrobotics.org>
Signed-off-by: Ivan Santiago Paunovic <ivanpauno@ekumenlabs.com> Co-authored-by: Steve Peters <scpeters@openrobotics.org>
Signed-off-by: Ivan Santiago Paunovic <ivanpauno@ekumenlabs.com> Co-authored-by: Steve Peters <scpeters@openrobotics.org>
Similarly to #2949, it fixes another condition where
gzserver
hangs when shutting down.The main thread was interrepted by sigint and is waiting for
ConnectionManager::Stop()
to finish:That will never happen, because
gazebo::rendering::Scene::PreRender()
was interrupted while taking this mutex and the connection manager thread got stuck in the same mutex:i.e. this
while
loop:https://github.com/osrf/gazebo/blob/2df3bfb039c1f5616122a0452ece99a8f3395e17/gazebo/transport/ConnectionManager.cc#L308-L316
will get stuck in L310 and never reach L316.
The solution is that
Stop()
will not wait until the background thread exits the loops, but just signal its ending.I also deleted the condition variable
notify_all()
call, as that's not signal safe.There's only a timed wait, so notifying or not shouldn't matter.