mgr: update mon metadata when monmap is updated #38932
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tchaikov
force-pushed
the
wip-48905
branch
2 times, most recently
from
5c5ae33
to
e0bdf4c
Compare
jdurgin
approved these changes
Jan 18, 2021
tchaikov
commented
Jan 18, 2021
there is chance that some monitor(s) is updated / upgraded in a single monmap update without being removed from cluster state's metata first, so, without this change, we will not update the metadata associated with that monitor, hence the mgr modules which consumes the metadata is not updated accordingly and keep reporting the stale information. in this change, we always update the metadata associated with all monitor included by the latest monmap. multiple "mon metadata" commands are sent to monitor for retrieving their updated metadata, instead of sending a single one, so that we can reuse "MetadataUpdate" to update the metadata of a given daemon. as the number of monitors in a typical cluster is relatively small, and the frequency of monmap update is low, so this overhead should be fine. unlike other places where we ask mon for metadata in Mgr class, the code sending the mon command for updated monitor metata is located outside of `cluster_state.with_monmap()` block, the reason is that `with_monmap()` is guraded by the monc_lock under the hood, while `start_mon_command()` also need to acquire the monc_lock, which is not a recursive lock. so we have to do this out of the `with_monmap()` block. Fixes: https://tracker.ceph.com/issues/48905 Signed-off-by: Kefu Chai <kchai@redhat.com>
This was referenced Jan 26, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
there is chance that some monitor(s) is updated / upgraded in a single
monmap update without being removed from cluster state's metata first,
so, without this change, we will not update the metadata associated with
that monitor, hence the mgr modules which consumes the metadata is not
updated accordingly and keep reporting the stale information.
in this change, we always update the metadata associated with all monitor
included by the latest monmap. multiple "mon metadata" commands are sent
to monitor for retrieving their updated metadata, instead of sending a
single one, so that we can reuse "MetadataUpdate" to update the metadata
of a given daemon. as the number of monitors in a typical cluster is
relatively small, and the frequency of monmap update is low, so this
overhead should be fine.
Fixes: https://tracker.ceph.com/issues/48905
Signed-off-by: Kefu Chai kchai@redhat.com
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume tox