New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimic: mds: optimize the way how max export size is enforced #23952

Merged
merged 8 commits into from Oct 19, 2018

Conversation

Projects
None yet
5 participants
@ukernel
Copy link
Member

ukernel commented Sep 6, 2018

https://tracker.ceph.com/issues/32100

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

ukernel added some commits Jul 16, 2018

mds: cleanup MutationImpl::More initialization
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit efff5c6)
mds: make CInode::get_dirfrags container agnostic
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 6538dae)
mds: optimize the way how max export size is enforced
The old way is checking export size after subtree gets frozen. It may
freeze a large subtree, but only exports small portion of the subtree.

The new way is choosing some subdirs according to the max export size,
then freeze these subdirs.

http://tracker.ceph.com/issues/25131
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c163777)

 Conflicts:
	src/mds/Migrator.cc
	src/mds/Migrator.h
mds: change default mds_max_export_size to 20M
1G seems too large. mds may speed long time on processing
imported/expored subtree.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 07ef150)

 Conflicts:
	src/common/options.cc
mds: limit total size of exporting subtrees
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit dd9ad34)

 Conflicts:
	src/mds/Migrator.cc
	src/mds/Migrator.h
mds: export subtree part by part
When exporting a large subtree, migrator may only exports some portions
of the subtree. This patch makes migrator continue to export the rest
partions when previous operations finish.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit da7fc1a)
mds: clear scatterlock's flushed when removing dirty
If mds imports scatterlock's parent inode later, the leftover flushed
flag may prevent Locker::eval_gather from calling scatter_writebehind
and result incorrect dirstat/neststat.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 2788b13)

@ukernel ukernel added the cephfs label Sep 6, 2018

mds: adjust export size after renaming directory into freezing subtree
restart exporting the subtree if rename makes export size too large

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit a7076fa)

 Conflicts:
	src/mds/MDSRank.h

@ukernel ukernel force-pushed the ukernel:mimic-mds-export-size branch from f114deb to 710a5f5 Sep 6, 2018

@smithfarm smithfarm added this to the mimic milestone Sep 6, 2018

@theanalyst

This comment has been minimized.

Copy link
Member

theanalyst commented Oct 2, 2018

Can you explain how the conflicts were resolved in the commits , it usually helps a lot in the future when looking at the commit why the backport was non-trivial

@yuriw

This comment has been minimized.

Copy link
Contributor

yuriw commented Oct 15, 2018

@yuriw yuriw merged commit 34602bc into ceph:mimic Oct 19, 2018

4 checks passed

Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment