mon/Monitor: fix statfs handling before luminous switchover happens #19243
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Is there a ticket where I can read more about this? I think I get what's happening, but as you say it's an ugly hack... |
|
should fix http://tracker.ceph.com/issues/21930 |
|
the ticket doesn't have much info. you can log into a test machine in this state, though see smithi185, smithi110 for the mons, and smithi145 for the hung client 2017-11-29 21:09:29.694437 7f54aabd8700 1 -- 172.21.15.30:0/283705467 --> 172.21.15.185:6789/0 -- statfs(23901 v26) v1 -- ?+0 0x55b54d886c00 con 0x55b54d528300 ... and the mons stuck with 2017-11-29 22:02:17.518638 7ff719fee700 10 mon.a@1(peon).paxosservice(mgrstat 25..25) dispatch 0x56439dbc0900 statfs(23900 pool -1 v26) v1 from client.14098 172.21.15.30:0/283705467 con 0x56439d490800 2017-11-29 22:02:17.518646 7ff719fee700 10 mon.a@1(peon).paxosservice(mgrstat 25..25) waiting for paxos -> readable (v26) 2017-11-29 22:02:17.518651 7ff719fee700 10 mon.a@1(peon).paxosservice(mgrstat 25..25) dispatch 0x56439ce5fa80 statfs(23901 pool -1 v26) v1 from client.14098 172.21.15.30:0/283705467 con 0x56439d490800 2017-11-29 22:02:17.518656 7ff719fee700 10 mon.a@1(peon).paxosservice(mgrstat 25..25) waiting for paxos -> readable (v26) |
After the mons are luminous but before we switch over to using the MgrStatMonitor's new info, the version on mgrstat will generally be << than that of pgmon, and the client will send that version with the request. This means that the statfs message will perpetually appear to be in the future and fail the is_readable() check. Fix this with any ugly hack that resets the version to 1 if we haven't completed the luminous upgrade yet. Signed-off-by: Sage Weil <sage@redhat.com>
liewegas
force-pushed
the
wip-fix-statfs-luminous
branch
from
6536388
to
c1c539f
Compare
gregsfortytwo
approved these changes
Nov 29, 2017
There was a problem hiding this comment.
Okay, seems like a simple enough fix for that.
Reviewed-by: Greg Farnum gfarnum@redhat.com =
batrick
approved these changes
Nov 29, 2017
|
Do we still want this in 12.2.2? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
After the mons are luminous but before we switch over to using the
MgrStatMonitor's new info, the version on mgrstat will generally be <<
than that of pgmon, and the client will send that version with the
request. This means that the statfs message will perpetually appear to be
in the future and fail the is_readable() check.
Fix this with any ugly hack that resets the version to 1 if we haven't
completed the luminous upgrade yet.
Signed-off-by: Sage Weil sage@redhat.com