/proc/driver/nvidia/capabilities: no such file or directory: unknown

[jiangxiaobin96]

When I run docker run -it --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all image_name, there is no directory /proc/driver/nvidia/capabilities in container.
daemon.json

{
    "default-runtime": "nvidia",
    "live-restore": true,
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}

nvidia-container-runtime version

$ nvidia-container-runtime -v
runc version 1.1.10
commit: v1.1.10-0-g18a0cb0
spec: 1.0.2-dev
go: go1.20.12
libseccomp: 2.3.3

nvidia-container-toolkit version

NVIDIA Container Runtime Hook version 1.14.3
commit: 53b24618a542025b108239fe602e66e912b7d6e2

[elezar]

@jiangxiaobin96 to trigger the inclusion of /proc/driver/nvidia/capabilities you would need to trigger MIG config or monitor mode. For what reason do you need access to these capabilities in the contianer?

[jiangxiaobin96]

kind cluster with gpu followed by issue

[elezar]

Thanks for the clarification. For kind specifically, a workaround is to unmount the /proc/driver/nvidia tmpfs that is mounted by the nvidia-contianer-cli to mask proc in a contianer where devices are being injected.

We use this in our DRA driver here: https://github.com/NVIDIA/k8s-dra-driver/blob/702a05b98b145bdd7e2beb8543fa24c69e2e7330/demo/clusters/kind/scripts/create-kind-cluster.sh#L34

[jiangxiaobin96]

DONE! Thank you for your reply.