Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
OpenEBS jiva volume continues to accept reads even after disk loss (simulated) #1387
Is this a BUG REPORT or FEATURE REQUEST?
A disk failure/loss was simulated on a single-replica OpenEBS volume by performing a truncate command on the sparse file (volume-head-000.img) in the node where the replica was scheduled. Reads (via dd from the client node) on the volume were seen to continue on the iSCSI target - though the Controller logs showed that the iSCSI target was stopped with I/O errors due to loss of backend storage.
What you expected to happen:
The truncate command was used to create a sparse file of the same name with a minimal size of 1M, and the read requests were being received for offsets beyond the current size of the disk - which should have failed/errored out. However, these continued (possibly returning junk/invalid data)
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Detaching the external disk upon which the jiva replica is created causes the following logs:
However the replica itself continues to run (which is undesirable)
Workaround: A re-attach, remount & subsequent restart of the pod will ensure the data replica resyncs (if quorum is intact).
Note: Any crashes (or deletes), if they occur will cause the local mount point to be used for this replica.
referenced this issue
Jan 24, 2019
When the disk on which jiva replicas are created gets detached, any subsequent IOs on the disk returns EIO. PR#175 in jiva handles this by doing fatal of the replica, so that, when the disk gets attached, replica starts serving IOs.