pacific: krbd: make sure the device node is accessible after the mapping #39642
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We have always assumed this to be the case and users' scripts and orchestration tools have grown to depend on this. Let's add some enforcement, prompted by [1]: "I am running my Kubernetes worker node inside of an LXC container which doesn't benefit from the device node created by the kernel, so I'm using udev to create the /dev/rbd* device nodes inside of the LXC container." which, through the unfortunate interaction with ceph-csi rbd plugin, results in data loss for "volumeMode: Filesystem" PVs because it ends up recreating the filesystem every time the PV is attached to the pod: "When deleting the pod and re-creating it, I can see that the RBD image is indeed being reformatted. This seems to be because when blkid is being run to check if the image is formatted, the /dev/rbd* device has not yet been created by udev. By the time the code gets down to running mkfs, the device is there and the damage is done." [1] ceph/ceph-csi#1820 Fixes: https://tracker.ceph.com/issues/49410 Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit f6854ac)
idryomov
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have always assumed this to be the case and users' scripts and
orchestration tools have grown to depend on this. Let's add some
enforcement, prompted by [1]:
"I am running my Kubernetes worker node inside of an LXC container
which doesn't benefit from the device node created by the kernel, so
I'm using udev to create the /dev/rbd* device nodes inside of the LXC
container."
which, through the unfortunate interaction with ceph-csi rbd plugin,
results in data loss for "volumeMode: Filesystem" PVs because it ends
up recreating the filesystem every time the PV is attached to the pod:
"When deleting the pod and re-creating it, I can see that the RBD
image is indeed being reformatted. This seems to be because when
blkid is being run to check if the image is formatted, the /dev/rbd*
device has not yet been created by udev. By the time the code gets
down to running mkfs, the device is there and the damage is done."
[1] ceph/ceph-csi#1820
Fixes: https://tracker.ceph.com/issues/49410
Signed-off-by: Ilya Dryomov idryomov@gmail.com
(cherry picked from commit f6854ac)