centos: if a "host dev" exists bind mount it over /dev #115
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a work around for containerization systems that have a problem with us using the host's /dev directly but we need a dynamic dev populated for devices, lvs and gluster-block support. Eventually it would be nice to move away from needing the host's /dev but that day is not here yet. Signed-off-by: John Mulligan <jmulligan@redhat.com>
obnoxxx
approved these changes
Dec 19, 2018
nixpanic
requested changes
Dec 20, 2018
There was a problem hiding this comment.
Hmm, after deleting this broken pod (cause unknown) to restart it, everything comes up nicely in the new pod. Restarting the other glusterfs pods in the cluster did not show any problems; glusterd and gluster-blockd are functioning as they should. 👍
This looks good, and in a running container mounting the dev-from-host onto /dev works. However I created a container-image with this change for testing, and for some reason glusterd and other processes fail to start...
I don't know yet what the cause is, as the errors do not look related to this change at all?! Manually starting glusterd (-N = no daemon):
sh-4.2# glusterd -N
ERROR: failed to create logfile "/var/log/glusterfs/glusterd.log" (Permission denied)
ERROR: failed to open logfile /var/log/glusterfs/glusterd.log
nixpanic
approved these changes
Dec 20, 2018
nixpanic
added a commit
to nixpanic/gluster-kubernetes
that referenced
this pull request
Dec 27, 2018
The gluster-setup.sh script in the glusterfs-server container now chceks for the HOST_DEV_DIR environment variable and will 'mount --rbind' the alternative path to /dev from the host to /dev in the container. This is needed to allow access to *all* devices on the host, including ones that get dynamically allocated or hot-plugged. See-also: gluster/gluster-containers#115 Signed-off-by: Niels de Vos <ndevos@redhat.com>
nixpanic
added a commit
to nixpanic/openshift-ansible
that referenced
this pull request
Dec 31, 2018
The /dev bind-mount really seems to be needed for gluster-block (or actually tcmu-runner) and the dynamic allocation of /dev/uio<N> devices. However mounting the /dev from the host directly on /dev in the container fails with current versions of CRI-O. The glusterfs container image has been adjusted to take the HOST_DEV_DIR environment variable and 'mount --rbind' that path on /dev once the container starts. Updates: gluster/gluster-containers#115
|
When will a new image be published? @nixpanic fix for Gluster block on OpenShift 3.11 seems to rely on this fix to the Gluster container. |
|
On Wed, Jan 02, 2019 at 11:26:19PM -0800, Benjamin Merot wrote:
When will a new image be published? @nixpanic fix for Gluster block on OpenShift 3.11 seems to rely on this fix to the Gluster container.
The image should be there now, see https://lists.gluster.org/pipermail/gluster-infra/2019-January/005311.html
|
nixpanic
added a commit
to nixpanic/gluster-kubernetes
that referenced
this pull request
Jan 8, 2019
The gluster-setup.sh script in the glusterfs-server container now chceks for the HOST_DEV_DIR environment variable and will 'mount --rbind' the alternative path to /dev from the host to /dev in the container. This is needed to allow access to *all* devices on the host, including ones that get dynamically allocated or hot-plugged. See-also: gluster/gluster-containers#115 Signed-off-by: Niels de Vos <ndevos@redhat.com>
jarrpa
pushed a commit
to gluster/gluster-kubernetes
that referenced
this pull request
Jan 9, 2019
The gluster-setup.sh script in the glusterfs-server container now chceks for the HOST_DEV_DIR environment variable and will 'mount --rbind' the alternative path to /dev from the host to /dev in the container. This is needed to allow access to *all* devices on the host, including ones that get dynamically allocated or hot-plugged. See-also: gluster/gluster-containers#115 Signed-off-by: Niels de Vos <ndevos@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a work around for containerization systems that
have a problem with us using the host's /dev directly but we
need a dynamic dev populated for devices, lvs and gluster-block
support.
Eventually it would be nice to move away from needing the
host's /dev but that day is not here yet.
Note
This checks if something that looks like a "/dev" is mounted in the container and bind mounts (recursively) that over the existing /dev. I'm currently using the environment variable HOST_DEV_DIR and default of /mnt/host-dev. I'm not wedded to these names and welcome discussions over what the dfaults should be.
We may also want to make matching changes to our templates as needed.
This has been tested on a cluster using cri-o and both "exec -it/rsh" commands and gluster-block appeared to work with a hand executed version of this. Let's discuss if this is the best short term fix for these issues.