Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ReplicatedPG::failed_push: release read lock on failure #11914

Merged
merged 1 commit into from Nov 14, 2016
Merged

ReplicatedPG::failed_push: release read lock on failure #11914

merged 1 commit into from Nov 14, 2016

Conversation

tchaikov
Copy link
Contributor

and requeue the blocked ops.

Fixes: http://tracker.ceph.com/issues/17857
Signed-off-by: Kefu Chai kchai@redhat.com

@tchaikov tchaikov added this to the kraken milestone Nov 11, 2016
@tchaikov tchaikov force-pushed the wip-17857 branch 2 times, most recently from 0cda9ce to 38a3bf4 Compare November 11, 2016 16:26
@tchaikov tchaikov changed the title ReplicatedPG::failed_push: release read lock on failure [DNM] ReplicatedPG::failed_push: release read lock on failure Nov 11, 2016
@athanatos
Copy link
Contributor

Yep, that'll do it. LGTM

liewegas
liewegas reviewed Nov 11, 2016
View reviewed changes
src/osd/ReplicatedPG.cc
assert(recovering.count(soid));
auto obc = recovering[soid];
if (obc) {
list<OpRequestRef> blocked_ops;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this if looks redundant if we assert count(soid)?

@dzafman dzafman mentioned this pull request Nov 12, 2016
Merged
@tchaikov
Copy link
Contributor Author 

26191: /home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:906: objectstore_tool:  ceph-objectstore-tool --data-path testdir/osd-scrub-repair/5 --journal-path testdir/osd-scrub-repair/5/journal SOMETHING list-attrs
26191: No object id 'SOMETHING' found or invalid JSON specified
26191: /home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:908: objectstore_tool:  return 1

jenkins is not happy because "ceph-objectstore-tool" failed.

@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 13, 2016 •
edited

ctest -R test-erasure-eio.sh --repeat-until-fail 20 i ran this test for 20 times locally. all of them passed. weird...

@tchaikov tchaikov force-pushed the wip-17857 branch 3 times, most recently from 1a4313f to f6bcf8f Compare November 14, 2016 05:14
@tchaikov
Copy link
Contributor Author

ctest -R test-erasure-eio.sh on jenkins passes. let's run the full "make check".

@tchaikov
ReplicatedPG::failed_push: release read lock on failure
b3224a1 
and requeue the blocked ops.

Fixes: http://tracker.ceph.com/issues/17857
Signed-off-by: Kefu Chai <kchai@redhat.com>
@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 14, 2016 •
edited 

    136 - osd-scrub-repair.sh (Failed)

20816: /home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:905: objectstore_tool:  ceph-objectstore-tool --debug --data-path testdir/osd-scrub-repair/5 --journal-path testdir/osd-scrub-repair/5/journal SOMETHING list-attrs
20816: 2016-11-14 11:31:07.295557 7fd0c997f980  0 filestore(testdir/osd-scrub-repair/5) backend generic (magic 0xef53)
20816: 2016-11-14 11:31:07.295580 7fd0c997f980 -1 filestore(testdir/osd-scrub-repair/5) WARNING: max attr value size (1024) is smaller than osd_max_object_name_len (2048).  Your backend filesystem appears to not support attrs large enough to handle the configured max rados name size.  You may get unexpected ENAMETOOLONG errors on rados operations or buggy behavior
20816: 2016-11-14 11:31:07.296024 7fd0c997f980  0 genericfilestorebackend(testdir/osd-scrub-repair/5) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
20816: 2016-11-14 11:31:07.296043 7fd0c997f980  0 genericfilestorebackend(testdir/osd-scrub-repair/5) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
20816: 2016-11-14 11:31:07.296045 7fd0c997f980  0 genericfilestorebackend(testdir/osd-scrub-repair/5) detect_features: splice() is disabled via 'filestore splice' config option
20816: 2016-11-14 11:31:07.351041 7fd0c997f980  0 genericfilestorebackend(testdir/osd-scrub-repair/5) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
20816: 2016-11-14 11:31:07.358997 7fd0c997f980  0 filestore(testdir/osd-scrub-repair/5) limited size xattrs
20816: 2016-11-14 11:31:07.359199 7fd0c997f980  0 filestore(testdir/osd-scrub-repair/5) start omap initiation
20816: 2016-11-14 11:31:07.364576 7fd0c997f980  0 filestore(testdir/osd-scrub-repair/5) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
20816: 2016-11-14 11:31:07.364753 7fd0c997f980 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
20816: 2016-11-14 11:31:07.364755 7fd0c997f980  1 journal _open testdir/osd-scrub-repair/5/journal fd 12: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
20816: 2016-11-14 11:31:07.365059 7fd0c997f980  1 journal _open testdir/osd-scrub-repair/5/journal fd 12: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
20816: 2016-11-14 11:31:07.365276 7fd0c997f980  1 filestore(testdir/osd-scrub-repair/5) upgrade
20816: Cluster fsid=bf52c831-444b-4855-9243-b97f664de1fd
20816: Supported features: compat={},rocompat={},incompat={1=initial feature set(~v.18),2=pginfo object,3=object locator,4=last_epoch_clean,5=categories,6=hobjectpool,7=biginfo,8=leveldbinfo,9=leveldblog,10=snapmapper,11=sharded objects,12=transaction hints,13=pg meta object,14=explicit missing set,15=fastinfo pg attr}
20816: On-disk features: compat={},rocompat={},incompat={1=initial feature set(~v.18),2=pginfo object,3=object locator,4=last_epoch_clean,5=categories,6=hobjectpool,7=biginfo,8=leveldbinfo,9=leveldblog,10=snapmapper,11=sharded objects,12=transaction hints,13=pg meta object,14=explicit missing set,15=fastinfo pg attr}
20816: 0 pgs to scan
20816: No object id 'SOMETHING' found or invalid JSON specified
20816: 2016-11-14 11:31:07.366006 7fd0c997f980  1 journal close testdir/osd-scrub-repair/5/journal
20816: /home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:908: objectstore_tool:  return 1

@tchaikov tchaikov changed the title [DNM] ReplicatedPG::failed_push: release read lock on failure ReplicatedPG::failed_push: release read lock on failure Nov 14, 2016
@tchaikov tchaikov merged commit 1a9c3d6 into ceph:master Nov 14, 2016
@tchaikov tchaikov deleted the wip-17857 branch November 14, 2016 16:02
@dzafman dzafman mentioned this pull request Dec 7, 2016
Closed
@dzafman dzafman mentioned this pull request Feb 1, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liewegas liewegas liewegas approved these changes

Assignees
No one assigned
Labels
bug-fix core
Projects
None yet
Milestone
kraken
4 participants
@tchaikov @athanatos @liewegas @dzafman