Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tools/cephfs: Use sharded pgls in cephfs-data-scan #7034

Merged
merged 3 commits into from Feb 18, 2016
Merged

tools/cephfs: Use sharded pgls in cephfs-data-scan #7034

merged 3 commits into from Feb 18, 2016

Conversation

jcsp
Copy link
Contributor

@jcsp jcsp commented Dec 23, 2015

No description provided.

@jcsp jcsp added feature cephfs Ceph File System labels Dec 23, 2015
@jcsp jcsp changed the title Use shareded pgls in cephfs-data-scan Use sharded pgls in cephfs-data-scan Dec 23, 2015
@jcsp jcsp changed the title Use sharded pgls in cephfs-data-scan tools/cephfs: Use sharded pgls in cephfs-data-scan Dec 23, 2015
@jcsp jcsp mentioned this pull request Dec 23, 2015
Merged
@jcsp
Copy link
Contributor Author

jcsp commented Dec 23, 2015

Is tested by: ceph/ceph-qa-suite#776

@jcsp
Copy link
Contributor Author

jcsp commented Jan 4, 2016

should we use metadata_io here?

Good catch, thank you. I've updated it.

@jcsp
Copy link
Contributor Author

jcsp commented Jan 6, 2016

Updated to fix the place where it was using data_io instead of ioctx in forall_objects (thanks zheng)

@jcsp
Copy link
Contributor Author

jcsp commented Jan 7, 2016

Passing tests here: http://pulpito.ceph.com/jspray-2016-01-06_16:00:46-fs-wip-jcsp-testing---basic-smithi/15866

(other failures in that suite are all either infrastructure or attributed to other patches)

gregsfortytwo added a commit that referenced this pull request Jan 13, 2016
@gregsfortytwo
Merge branch 'wip-sharded-scan' of git://github.com/jcsp/ceph into gr…
d0a50c9 
…eg-fs-speculative

#7034

Conflicts:
	src/CMakeLists.txt
Just adding two new binaries in same line
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
@jcsp
Copy link
Contributor Author

jcsp commented Jan 14, 2016

Note to self: failure here: http://pulpito.ceph.com/gregf-2016-01-13_16:10:54-fs-greg-fs-speculative-1---basic-mira/28336/

@jcsp
Copy link
Contributor Author

jcsp commented Jan 29, 2016

Rebased

@gregsfortytwo
Copy link
Member

@jcsp, @ukernel, does this need more review for the changes? Ready for integration testing?

@jcsp
Copy link
Contributor Author

jcsp commented Feb 2, 2016

Although I've got a passing test run here: http://pulpito.ceph.com/jspray-2016-01-29_03:42:31-fs-wip-sharded-scan---basic-mira/ ...

... I gave it another couple of passes locally and there's an issue, so hold fire.

John Spray added 3 commits February 5, 2016 12:56
cmake: add cls_cephfs to cephfs_testing target
e4abb95 
Signed-off-by: John Spray <john.spray@redhat.com>
librados: enable filtering in object_list API
168e142 
Signed-off-by: John Spray <john.spray@redhat.com>
tools/cephfs: sharded pgls in cephfs-data-scan
bacfce5 
Signed-off-by: John Spray <john.spray@redhat.com>
@jcsp
Copy link
Contributor Author

jcsp commented Feb 5, 2016

This problem was that find_or_create_dirfrag wasn't handling EEXIST properly, so when two workers tried to create the same frag concurrently one of them was erroring out. Fixed now, test passes locally, ready for re-test.

@gregsfortytwo
Copy link
Member

I've got this in a testing branch but the unit test builder there failed on some monitor thing...probably just noise or a broken master but FYI.

gregsfortytwo added a commit that referenced this pull request Feb 8, 2016
@gregsfortytwo
Merge branch 'wip-sharded-scan' of git://github.com/jcsp/ceph into gr…
8c1fb20 
…eg-fs-testing

#7034

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Conflicts:
	src/tools/cephfs/DataScan.cc
gregsfortytwo added a commit that referenced this pull request Feb 15, 2016
@gregsfortytwo
Merge branch 'wip-sharded-scan' of git://github.com/jcsp/ceph into gr…
adb797a 
…eg-fs-testing

#7034

Conflicts:
	src/osdc/Objecter.cc
gregsfortytwo added a commit that referenced this pull request Feb 15, 2016
@gregsfortytwo
Merge branch 'wip-sharded-scan' of git://github.com/jcsp/ceph into gr…
374b55d 
…eg-fs-testing

#7034

Conflicts:
	src/osdc/Objecter.cc
@gregsfortytwo
Copy link
Member

http://pulpito.ceph.com/gregf-2016-02-17_22:27:35-fs-greg-pr-7034---basic-mira/
Valgrind failure is known (http://tracker.ceph.com/issues/14794)
Other failure is infrastructure noise; that test has passed previous integration branches including this code.

@gregsfortytwo gregsfortytwo merged commit bacfce5 into ceph:master Feb 18, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@jcsp jcsp

Labels
cephfs Ceph File System feature needs-qa wip-greg-testing
Projects
None yet
Milestone
No milestone
2 participants
@jcsp @gregsfortytwo