New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

luminous: mds: optimize the way how max export size is enforced #23789

Merged
merged 9 commits into from Sep 19, 2018

Conversation

Projects
None yet
5 participants
@ukernel
Member

ukernel commented Aug 28, 2018

ukernel added some commits Jul 12, 2018

mds: change default mds_max_export_size to 20M
1G seems too large. mds may speed long time on processing
imported/expored subtree.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
mds: cleanup MutationImpl::More initialization
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 90fd9c8)

 Conflicts:
	src/mds/Mutation.h
mds: limit total size of exporting subtrees
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
mds: export subtree part by part
When exporting a large subtree, migrator may only exports some portions
of the subtree. This patch makes migrator continue to export the rest
partions when previous operations finish.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
mds: make CInode::get_dirfrags container agnostic
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c2807b8)

 Conflicts:
	src/mds/CInode.cc
	src/mds/CInode.h
mds: clear scatterlock's flushed when removing dirty
If mds imports scatterlock's parent inode later, the leftover flushed
flag may prevent Locker::eval_gather from calling scatter_writebehind
and result incorrect dirstat/neststat.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
mds: adjust export size after renaming directory into freezing subtree
restart exporting the subtree if rename makes export size too large

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
mds: optimize the way how max export size is enforced
The old way is checking export size after subtree gets frozen. It may
freeze a large subtree, but only exports small portion of the subtree.

The new way is choosing some subdirs according to the max export size,
then freeze these subdirs.

http://tracker.ceph.com/issues/25131
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

@ukernel ukernel added the cephfs label Aug 28, 2018

@smithfarm smithfarm added this to the luminous milestone Aug 29, 2018

@smithfarm smithfarm added feature and removed documentation labels Aug 29, 2018

@smithfarm smithfarm requested a review from batrick Aug 29, 2018

@yuriw

This comment has been minimized.

Contributor

yuriw commented Sep 13, 2018

@batrick

This comment has been minimized.

Member

batrick commented Sep 18, 2018

Needs rebase. Testing passed.

@yuriw

This comment has been minimized.

Contributor

yuriw commented Sep 18, 2018

@ukernel pls merge after rebase

@ukernel ukernel merged commit fc9cc68 into ceph:luminous Sep 19, 2018

0 of 4 checks passed

Docs: build check Docs: building
Details
Signed-off-by checking if commits are signed
Details
Unmodified Submodules checking if PR has modified submodules
Details
make check running make check
Details

@ukernel ukernel deleted the ukernel:luminous-mds-export-size branch Sep 19, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment