Skip to content

DevicePlugin not able to contact kubelet, missing volume mount in containerized openshift (oc cluster up) #20633

@zvonkok

Description

@zvonkok

DevicePlugin not able to contact kubelet because of the missing volume in containerized install
of openshift (oc cluster up).

The missing volume is /var/lib/kubelet/device-plugins.

Version
oc v3.11.0-alpha.0+5009149-867
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://127.0.0.1:8443
openshift v3.11.0-alpha.0+5009149-867
kubernetes v1.11.0+d4cacc0
Steps To Reproduce

oc cluster up

https://blog.openshift.com/how-to-use-gpus-with-deviceplugin-in-openshift-3-10/

Current Result
018/08/14 08:03:52 Loading NVML
2018/08/14 08:03:52 Fetching devices.
2018/08/14 08:03:52 Starting FS watcher.
2018/08/14 08:03:52 Starting OS watcher.
2018/08/14 08:03:52 Starting to serve on /var/lib/kubelet/device-plugins/nvidia.sock
2018/08/14 08:03:57 Could not register device plugin: context deadline exceeded
2018/08/14 08:03:57 Could not contact Kubelet, retrying. Did you enable the device plugin feature gate?
2018/08/14 08:03:57 You can check the prerequisites at: https://github.com/NVIDIA/k8s-device-plugin#prerequisites
2018/08/14 08:03:57 You can learn how to set the runtime at: https://github.com/NVIDIA/k8s-device-plugin#quick-start
Expected Result

The following result is gathered when the /var/lib/kubelet/device-plugins is mounted into
openshift/origin-node.

2018/08/14 09:55:39 Loading NVML
2018/08/14 09:55:40 Fetching devices.
2018/08/14 09:55:40 Starting FS watcher.
2018/08/14 09:55:40 Starting OS watcher.
2018/08/14 09:55:40 Starting to serve on /var/lib/kubelet/device-plugins/nvidia.sock
2018/08/14 09:55:40 Registered device plugin with Kubelet
Additional Information

The DevicePlugin registers itself with the kubelet through the Unix socket at host path /var/lib/kubelet/device-plugins/kubelet.sock, which is missing if the kubelet is started in a container
without the volume mount.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions