mds: misc load balancer fixes #19220
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ukernel
force-pushed
the
wip-mds-balancer
branch
from
b1bc1a0
to
1993254
Compare
ukernel
force-pushed
the
wip-mds-balancer
branch
from
b5d69e0
to
9a3e1eb
Compare
ukernel
force-pushed
the
wip-mds-balancer
branch
from
9a3e1eb
to
f5a7efb
Compare
ukernel
force-pushed
the
wip-mds-balancer
branch
from
f5a7efb
to
3356b15
Compare
ukernel
force-pushed
the
wip-mds-balancer
branch
3 times, most recently
from
2f2c3ae
to
527d746
Compare
|
Would you please file a tracker.ceph.com ticket to ensure this is backported to luminous? |
ukernel
force-pushed
the
wip-mds-balancer
branch
5 times, most recently
from
527d746
to
6d3d5a5
Compare
Signed-off-by: "Yan, Zheng" <zyan@redhat.com> Fixes: http://tracker.ceph.com/issues/21745
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Changing load counters while printing load results confusing output. Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
MDCache::trim() already does the same job Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
There were codes that pass mdr->get_mds_stamp() to {hit_inode,hit_dir}.
The stamp can be older than 'last_decay' in decay counters, which can
cause DecayCounter::decay() to malfunction.
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
no logic changes Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Migrator will refuse to export them. Besides, stray directory can never be subtree root. Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
It doesn't make sense to call MDBalancer::find_exports() for a dirfrag that is already being exported. Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
1. change import_pop_map to multimap because subtrees may have the same popularity. 2. avoid calculating subtrees' popularity multiple times Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
When choosing dirfrags for exporting, consider subtrees imported from other MDS first. It's because that exporting dirfrags chosen by MDBalancer::find_export() will create new subtrees. Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
stop at subtree bounds; make freezing/frozen check more efficient. Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
using map::insert to access map items looks strang Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
No need to get this information from MHeartbeat Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
MDBalancer::find_exports() can be more efficient with LRU list. Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
ukernel
force-pushed
the
wip-mds-balancer
branch
from
6d3d5a5
to
fc0d340
Compare
|
retest this please |
batrick
approved these changes
Apr 11, 2018
batrick
added a commit
that referenced
this pull request
Apr 11, 2018
* refs/pull/19220/head:
mds: limit run time of load balancer
mds: add list to track recently used sub-directories
mds: calculate other mds' last_epoch_under locally
mds: cleanup mds_load map access/update
mds: check export pin when choosing dirfrags for exporting
mds: optimize MDBalancer::find_exports()
mds: avoid creating unnecessary subtrees during load balance
mds: mds: optimize MDBalancer::try_rebalance()
mds: don't try exporting subdir if dirfrag is already being exported
mds: don't try exporting dirfrags under mds's own mdsdir
mds: cleanup MDBalancer::try_rebalance()
mds: always pass current time to MDBalancer::{hit_inode,hit_dir}
mds: remove unused MDBalancer::export_empties()
mds: don't decay load counters when printing load
mds: adjust pop_nested when importing subtree
mds: adjust subtree popularity when import fails
mds: adjust subtree popularity after rename
mds: fix request rate calculation
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.