Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimic: mds: behind on trimming and [dentry] was purgeable but no longer is! #29224

Merged
merged 2 commits into from Oct 17, 2019

Conversation

@smithfarm
Copy link
Contributor

smithfarm commented Jul 23, 2019

@smithfarm smithfarm self-assigned this Jul 23, 2019
@smithfarm smithfarm added this to the mimic milestone Jul 23, 2019
@smithfarm smithfarm added bug fix core cephfs and removed core labels Jul 23, 2019
@smithfarm smithfarm requested review from ukernel and batrick Jul 23, 2019
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Jul 23, 2019

/home/jenkins-build/build/workspace/ceph-pull-requests/src/mds/Migrator.cc:2661:7: error: 'C_MDS_ExportPrepFactory' was not declared in this scope
       C_MDS_ExportPrepFactory cf(this, m);
       ^~~~~~~~~~~~~~~~~~~~~~~
/home/jenkins-build/build/workspace/ceph-pull-requests/src/mds/Migrator.cc:2661:7: note: suggested alternative: 'C_MDC_ExportFreeze'
       C_MDS_ExportPrepFactory cf(this, m);
       ^~~~~~~~~~~~~~~~~~~~~~~
       C_MDC_ExportFreeze
/home/jenkins-build/build/workspace/ceph-pull-requests/src/mds/Migrator.cc:2662:27: error: 'cf' was not declared in this scope
       gather.set_finisher(cf.build());
                           ^~
src/mds/CMakeFiles/mds.dir/build.make:398: recipe for target 'src/mds/CMakeFiles/mds.dir/Migrator.cc.o' failed
make[3]: *** [src/mds/CMakeFiles/mds.dir/Migrator.cc.o] Error 1
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Jul 23, 2019

Hm, something missing here.

@smithfarm smithfarm force-pushed the smithfarm:wip-39223-mimic branch from 333aebc to 87badcd Jul 23, 2019
@@ -2656,6 +2657,12 @@ void Migrator::handle_export_prep(MExportDirPrep *m)
}
}

if (gather.has_subs()) {
gather.set_finisher(new C_M_ExportSessionsFlushed(this, dir, it->second.tid));

This comment has been minimized.

Copy link
@smithfarm

smithfarm Jul 24, 2019

Author Contributor

This is how I fixed the build failure - it was a "stab in the dark", but it seems to have done the trick.

This comment has been minimized.

Copy link
@batrick

batrick Oct 7, 2019

Member

This should be: gather.set_finisher(new C_MDS_RetryMessage(mds, m));

This caused:

2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.cache discover_reply discover_reply(2 0x20000000001) v2
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.cache  found tid 2
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.cache add_replica_dir added [dir 0x20000000001 /1/4/5/ [2,head] rep@1.1 state=8192|sticky f() n() hs=0+0,ss=0+0 | sticky=1 0x55f0d614ae00] nonce 1
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.cache.ino(0x20000000001) take_dir_waiting frag * on [inode 0x20000000001 [...2,head] /1/4/5/ rep@1.1 v4 f() n(v0 rc2019-10-05 18:58:37.097781 1=0+1) (iauth lock) (ifile lock) (ixattr lock) (iversion lock) | dirfrag=1 stickydirs=1 dirwaiter=1 discoverbase=0 export_pin=1 0x55f0d614a700]
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.19 mds has 1 queued contexts
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.19 0x55f0d5b1a7d0
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.19  finish 0x55f0d5b1a7d0
2019-10-05 18:58:56.474 7f37b9a91700 10 MDSInternalContextBase::complete: 25C_M_ExportSessionsFlushed
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.migrator export_sessions_flushed [dir 0x10000000000 /1/ [2,head] rep@1.2 dir_auth=1 state=264192|importing f(v1 m2019-10-05 18:58:36.934747 2=0+2) n(v5 rc2019-10-05 18:58:37.097781 4=0+4) hs=2+0,ss=0+0 | dnwaiter=0 child=1 frozen=0 subtree=1 importing=1 replicated=0 dirty=0 waiter=0 authpin=0 tempexporting=0 0x55f0d5eda700]
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.migrator export must have aborted on 0x55f0d5eda700

/ceph/teuthology-archive/yuriw-2019-10-05_14:50:45-multimds-wip-yuri-testing-2019-10-04-1441-mimic-testing-basic-smithi/4362360/remote/smithi144/log/ceph-mds.b.log.gz

For failure:

2019-10-05T18:59:23.451 INFO:tasks.cephfs_test_runner:======================================================================
2019-10-05T18:59:23.451 INFO:tasks.cephfs_test_runner:ERROR: test_export_pin (tasks.cephfs.test_exports.TestExports)
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri-testing-2019-10-04-1441-mimic/qa/tasks/cephfs/test_exports.py", line 81, in test_export_pin
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:    self._wait_subtrees(status, 0, [('/1', 0), ('/1/2', 1), ('/1/4/5', 1)])
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri-testing-2019-10-04-1441-mimic/qa/tasks/cephfs/test_exports.py", line 27, in _wait_subtrees
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:    raise RuntimeError("rank {0} failed to reach desired subtree state", rank)
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:RuntimeError: ('rank {0} failed to reach desired subtree state', 0)

From: /ceph/teuthology-archive/yuriw-2019-10-05_14:50:45-multimds-wip-yuri-testing-2019-10-04-1441-mimic-testing-basic-smithi/4362360/teuthology.log

@yuriw

This comment has been minimized.

Copy link
Contributor

yuriw commented Aug 7, 2019

@@ -2656,6 +2657,12 @@ void Migrator::handle_export_prep(MExportDirPrep *m)
}
}

if (gather.has_subs()) {
gather.set_finisher(new C_M_ExportSessionsFlushed(this, dir, it->second.tid));

This comment has been minimized.

Copy link
@batrick

batrick Oct 7, 2019

Member

This should be: gather.set_finisher(new C_MDS_RetryMessage(mds, m));

This caused:

2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.cache discover_reply discover_reply(2 0x20000000001) v2
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.cache  found tid 2
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.cache add_replica_dir added [dir 0x20000000001 /1/4/5/ [2,head] rep@1.1 state=8192|sticky f() n() hs=0+0,ss=0+0 | sticky=1 0x55f0d614ae00] nonce 1
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.cache.ino(0x20000000001) take_dir_waiting frag * on [inode 0x20000000001 [...2,head] /1/4/5/ rep@1.1 v4 f() n(v0 rc2019-10-05 18:58:37.097781 1=0+1) (iauth lock) (ifile lock) (ixattr lock) (iversion lock) | dirfrag=1 stickydirs=1 dirwaiter=1 discoverbase=0 export_pin=1 0x55f0d614a700]
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.19 mds has 1 queued contexts
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.19 0x55f0d5b1a7d0
2019-10-05 18:58:56.474 7f37b9a91700 10 mds.0.19  finish 0x55f0d5b1a7d0
2019-10-05 18:58:56.474 7f37b9a91700 10 MDSInternalContextBase::complete: 25C_M_ExportSessionsFlushed
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.migrator export_sessions_flushed [dir 0x10000000000 /1/ [2,head] rep@1.2 dir_auth=1 state=264192|importing f(v1 m2019-10-05 18:58:36.934747 2=0+2) n(v5 rc2019-10-05 18:58:37.097781 4=0+4) hs=2+0,ss=0+0 | dnwaiter=0 child=1 frozen=0 subtree=1 importing=1 replicated=0 dirty=0 waiter=0 authpin=0 tempexporting=0 0x55f0d5eda700]
2019-10-05 18:58:56.474 7f37b9a91700  7 mds.0.migrator export must have aborted on 0x55f0d5eda700

/ceph/teuthology-archive/yuriw-2019-10-05_14:50:45-multimds-wip-yuri-testing-2019-10-04-1441-mimic-testing-basic-smithi/4362360/remote/smithi144/log/ceph-mds.b.log.gz

For failure:

2019-10-05T18:59:23.451 INFO:tasks.cephfs_test_runner:======================================================================
2019-10-05T18:59:23.451 INFO:tasks.cephfs_test_runner:ERROR: test_export_pin (tasks.cephfs.test_exports.TestExports)
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri-testing-2019-10-04-1441-mimic/qa/tasks/cephfs/test_exports.py", line 81, in test_export_pin
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:    self._wait_subtrees(status, 0, [('/1', 0), ('/1/2', 1), ('/1/4/5', 1)])
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri-testing-2019-10-04-1441-mimic/qa/tasks/cephfs/test_exports.py", line 27, in _wait_subtrees
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:    raise RuntimeError("rank {0} failed to reach desired subtree state", rank)
2019-10-05T18:59:23.452 INFO:tasks.cephfs_test_runner:RuntimeError: ('rank {0} failed to reach desired subtree state', 0)

From: /ceph/teuthology-archive/yuriw-2019-10-05_14:50:45-multimds-wip-yuri-testing-2019-10-04-1441-mimic-testing-basic-smithi/4362360/teuthology.log

ukernel added 2 commits Mar 19, 2019
Fixes: http://tracker.ceph.com/issues/38679
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit f518577)
Fixes: http://tracker.ceph.com/issues/38679
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 07de312)
(cherry picked from commit 1f0dcf2)

Note:
        the first "(cherry picked from . . .)" line is present in the master
        commit 1f0dcf2

Conflicts:
	src/mds/Migrator.cc
- no C_MDS_ExportPrepFactory in mimic
@smithfarm smithfarm force-pushed the smithfarm:wip-39223-mimic branch from 87badcd to ea214f5 Oct 8, 2019
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Oct 8, 2019

@batrick I made the change you requested, but the second commit still looks very different from its counterpart in master...

@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Oct 8, 2019

Ah - I think I was trying to model it on @ukernel 's luminous commit 85d8970 so it should be OK now?

@batrick

This comment has been minimized.

Copy link
Member

batrick commented Oct 8, 2019

Ah - I think I was trying to model it on @ukernel 's luminous commit 85d8970 so it should be OK now?

Yes, looks right now. Thanks @smithfarm

@batrick batrick added the needs-qa label Oct 8, 2019
@batrick batrick self-requested a review Oct 8, 2019
@smithfarm smithfarm dismissed batrick’s stale review Oct 8, 2019

requested changed made and approved

@yuriw

This comment has been minimized.

Copy link
Contributor

yuriw commented Oct 13, 2019

@yuriw yuriw merged commit a283989 into ceph:mimic Oct 17, 2019
4 checks passed
4 checks passed
Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
@smithfarm smithfarm deleted the smithfarm:wip-39223-mimic branch Oct 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.