Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mds: check metadata pool not cluster is full #19602

Merged
merged 1 commit into from Dec 21, 2017
Merged

Conversation

batrick
Copy link
Member

@batrick batrick commented Dec 19, 2017

CEPH_OSDMAP_FULL flag was obsoleted by
b4ca5ae. So, check if the metadata pool is
full instead which is a decent proxy (as metadata operations can still
proceed). However, the data pool may still be full which would result in some
operations still not completing (like inode backtrace updates).

Fixes: http://tracker.ceph.com/issues/22483

Signed-off-by: Patrick Donnelly pdonnell@redhat.com

@batrick batrick added bug-fix cephfs Ceph File System DNM labels Dec 19, 2017
@batrick batrick force-pushed the i22483 branch 3 times, most recently from e33134d to 39d07b3 Compare December 19, 2017 20:12
@batrick batrick changed the title [DNM] mds: check metadata pool not cluster is full mds: check metadata pool not cluster is full Dec 19, 2017
@batrick batrick added needs-review and removed DNM labels Dec 19, 2017
@batrick batrick requested a review from ukernel December 19, 2017 21:56
@@ -1720,7 +1720,9 @@ void Server::handle_osd_map()
* using osdmap_full_flag(), because we want to know "is the flag set"
* rather than "does the flag apply to us?" */
mds->objecter->with_osdmap([this](const OSDMap& o) {
is_full = o.test_flag(CEPH_OSDMAP_FULL);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

auto pi = o.get_pg_pool(mds->mdsmap->get_metadata_pool());
is_full = pi && pi->has_flag(pg_pool_t::FLAG_FULL);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking for that and couldn't find the right thing. Thanks!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -1720,7 +1720,9 @@ void Server::handle_osd_map()
* using osdmap_full_flag(), because we want to know "is the flag set"
* rather than "does the flag apply to us?" */
mds->objecter->with_osdmap([this](const OSDMap& o) {
is_full = o.test_flag(CEPH_OSDMAP_FULL);
std::set<int64_t> full, backfillfull, nearfull;
o.get_full_pools(mds->cct, &full, &backfillfull, &nearfull);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is used by the mon to set/clear the pool flags (and is slow). we only need to watch the flags.

CEPH_OSDMAP_FULL flag was obsoleted by
b4ca5ae. So, check if the metadata pool is
full instead which is a decent proxy (as metadata operations can still
proceed). However, the data pool may still be full which would result in some
operations still not completing (like inode backtrace updates).

Fixes: http://tracker.ceph.com/issues/22483

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
@ukernel
Copy link
Contributor

ukernel commented Dec 20, 2017

The data pool full check is still missing?

batrick added a commit to batrick/ceph that referenced this pull request Dec 21, 2017
* refs/pull/19602/head:
	mds: check metadata pool not cluster is full
@liewegas liewegas merged commit c461d59 into ceph:mimic-dev1 Dec 21, 2017
@batrick batrick deleted the i22483 branch December 21, 2017 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants