New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimic: os/bluestore: handle spurious read errors #24647

Merged
merged 1 commit into from Oct 29, 2018

Conversation

Projects
None yet
5 participants
@liewegas
Copy link
Member

liewegas commented Oct 17, 2018

Some kernels (4.9+) sometime fail to return data when reading
from a block device under memory pressure. This patch retries
the read if the checksum verification fails, tests show that
the first retried read succeeds in ~99.5% of the cases, so
3 attempts are made by default before giving up on the data.

Works-around: http://tracker.ceph.com/issues/22464
Signed-off-by: Paul Emmerich paul.emmerich@croit.io
(cherry picked from commit cffcbc7)

  • conflict due to adjacent tests in store_test

@liewegas liewegas added this to the mimic milestone Oct 17, 2018

@liewegas liewegas requested a review from ifed01 Oct 17, 2018

@liewegas

This comment has been minimized.

Copy link
Member

liewegas commented Oct 17, 2018

}

cerr << "Injecting CRC error with no retry, expecting EIO" << std::endl;
SetVal(g_conf(), "bluestore_retry_disk_reads", "0");

This comment has been minimized.

@emmericp

emmericp Oct 17, 2018

Contributor

the test uses g_conf directly, not g_conf() in mimic, see our backport here: croit@35cb7d9

no idea about luminous, but I suspect it might have the same problem

os/bluestore: handle spurious read errors
Some kernels (4.9+) sometime fail to return data when reading
from a block device under memory pressure. This patch retries
the read if the checksum verification fails, tests show that
the first retried read succeeds in ~99.5% of the cases, so
3 attempts are made by default before giving up on the data.

Works-around: http://tracker.ceph.com/issues/22464
Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit cffcbc7)

- conflict due to adjacent tests in store_test
- g_conf, not g_conf()

@liewegas liewegas force-pushed the liewegas:wip-pr-23273-mimic branch from dd3a001 to 34a9104 Oct 17, 2018

@yuriw

This comment has been minimized.

Copy link
Contributor

yuriw commented Oct 23, 2018

@ifed01

ifed01 approved these changes Oct 24, 2018

@yuriw yuriw merged commit b20ee1a into ceph:mimic Oct 29, 2018

4 checks passed

Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment