Can't enable nvidia gpu

[zengzhengrong]

Before creating an issue, make sure you've checked the following:

• You are running the latest released version of k0s
• Make sure you've searched for existing issues, both open and closed
• Make sure you've searched for PRs too, a fix might've been merged already
• You're looking at docs for the released version, "main" branch docs are usually ahead of released versions.

Platform

Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Version

v1.23.9+k0s

Sysinfo

root@k0s:/# k0s sysinfo
KERNEL_VERSION: 5.10.102.1-microsoft-standard-WSL2
CONFIG_INET: enabled
CONFIG_NETFILTER_XT_TARGET_REDIRECT: enabled
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled
CONFIG_NAMESPACES: enabled
CONFIG_UTS_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_NET_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_PIDS: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CPUSETS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_MEMCG: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_FAIR_GROUP_SCHED: enabled
CONFIG_EXT4_FS: enabled
CONFIG_PROC_FS: enabled
CONFIG_OVERLAY_FS: enabled
CONFIG_BLK_DEV_DM: enabled
CONFIG_CFS_BANDWIDTH: enabled
CONFIG_CGROUP_HUGETLB: enabled
CONFIG_SECCOMP: enabled
CONFIG_SECCOMP_FILTER: enabled
CONFIG_BRIDGE: enabled
CONFIG_IP6_NF_FILTER: enabled
CONFIG_IP6_NF_IPTABLES: enabled
CONFIG_IP6_NF_MANGLE: enabled
CONFIG_IP6_NF_NAT: enabled
CONFIG_IP_NF_FILTER: enabled
CONFIG_IP_NF_IPTABLES: enabled
CONFIG_IP_NF_MANGLE: enabled
CONFIG_IP_NF_NAT: enabled
CONFIG_IP_NF_TARGET_REJECT: enabled
CONFIG_IP_SET: enabled
CONFIG_IP_SET_HASH_IP: enabled
CONFIG_IP_SET_HASH_NET: enabled
CONFIG_IP_VS_NFCT: enabled
CONFIG_LLC: enabled
CONFIG_NETFILTER_NETLINK: enabled
CONFIG_NETFILTER_XTABLES: enabled
CONFIG_NETFILTER_XT_MARK: enabled
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled
CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled
CONFIG_NETFILTER_XT_MATCH_MULTIPORT: enabled
CONFIG_NETFILTER_XT_MATCH_RECENT: not set
CONFIG_NETFILTER_XT_MATCH_STATISTIC: enabled
CONFIG_NETFILTER_XT_NAT: enabled
CONFIG_NETFILTER_XT_SET: enabled
CONFIG_NETFILTER_XT_TARGET_MASQUERADE: enabled
CONFIG_NF_CONNTRACK: enabled
CONFIG_NF_DEFRAG_IPV4: enabled
CONFIG_NF_DEFRAG_IPV6: enabled
CONFIG_NF_NAT: enabled
CONFIG_NF_REJECT_IPV4: enabled
CONFIG_STP: enabled
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: enabled

What happened?

1. k0s ctr --gpus 0 command does not useful gpu

k0s ctr run --gpus 0 --rm   nvcr.io/nvidia/k8s/cuda-sample:nbody test-gpu /tmp/nbody -gpu -benchmark
Error: exec: "containerd": executable file not found in $PATH

Use mv /var/lib/k0s/bin/containerd usr/local/bin/ can to solve above

k0s ctr run --gpus 0 --rm   nvcr.io/nvidia/k8s/cuda-sample:nbody test-gpu /tmp/nbody -gpu -benchmark
Run "nbody -benchmark [-numbodies=<numBodies>]" to measure performance.
        -fullscreen       (run n-body simulation in fullscreen mode)
        -fp64             (use double precision floating point values for simulation)
        -hostmem          (stores simulation data in host memory)
        -benchmark        (run benchmark to measure performance) 
        -numbodies=<N>    (number of bodies (>= 1) to run in simulation) 
        -device=<d>       (where d=0,1,2.... for the CUDA device to use)
        -numdevices=<i>   (where i=(number of CUDA devices > 0) to use for simulation)
        -compare          (compares simulation results running once on the default GPU and once on the CPU)
        -cpu              (run n-body simulation on the CPU)
        -tipsy=<file.bin> (load a tipsy model file for simulation)

NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.

> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
GPU Device 0: "Pascal" with compute capability 6.1

> Compute 6.1 CUDA device: [NVIDIA GeForce GTX 1060 3GB]
9216 bodies, total time for 10 iterations: 7.372 ms
= 115.216 billion interactions per second
= 2304.320 single-precision GFLOP/s at 20 flops per interaction

And I install k8s-device-plugin have this problem
#NVIDIA/k8s-device-plugin#332

Does have any simple way to enable gpu like microk8s

#https://microk8s.io/docs/nvidia-dgx