New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mimic: Fix ceph-mgr restarts #22051
mimic: Fix ceph-mgr restarts #22051
Conversation
The prometheus serve method uses block which it should not as it can cause several problems when shutting down (restarting) ceph-mgr. As a result the restart of the ceph-mgr module will block until time-out if the prometheus module is enabled. This fixes the problem by using a simple simple shutdown_event instead of block() and stopping the server afterwards. Signed-off-by: Boris Ranto <branto@redhat.com> (cherry picked from commit a662caa)
The dashboard serve method uses block which it should not as it can cause several problems when shutting down (restarting) ceph-mgr. As a result the restart of the ceph-mgr module will block until time-out if the dashboard module is enabled. This fixes the problem by using a simple simple shutdown_event instead of block() and stopping the server afterwards. Signed-off-by: Boris Ranto <branto@redhat.com>
The prometheus part lgtm. Adding @rjfd for the dashboard commit. |
@s0nea could you run the dashboard QA tests with this PR? (notice this is targeting mimic) |
@rjfd yes, I'm going to try it (I'm not really sure if this will be successful because of the issues I'm currently facing in Teuthology). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the prometheus part lgtm
The QA test run was successful: http://pulpito.ceph.com/tdehler-2018-05-17_12:16:04-rados:mgr-wip-tdehler-testing-distro-basic-mira/ |
That is great. @rjfd ok to merge? |
Possibly related: http://tracker.ceph.com/issues/24166 |
already tested. so no need to run rados qa suite again. |
This PR contains a mimic back-port for shutdown/restart fix for prometheus module and a patch to fix the shutdown/restart with dashboard module enabled in the similar way.