Skip to content
This repository has been archived by the owner on Jan 22, 2024. It is now read-only.

"The NVIDIA Driver was not detected. GPU functionality will not be available." while using the latest drivers and nvidia-docker #1548

Closed
8 of 9 tasks
PQLLUX opened this issue Sep 17, 2021 · 3 comments
Closed
8 of 9 tasks

"The NVIDIA Driver was not detected. GPU functionality will not be available." while using the latest drivers and nvidia-docker #1548

PQLLUX opened this issue Sep 17, 2021 · 3 comments

Comments

@PQLLUX
Copy link

PQLLUX commented Sep 17, 2021

1. Issue or feature description

According to release notes here in cuda on wsl2 documentation the issue with NGC containers displaying the following message The NVIDIA Driver was not detected. GPU functionality will not be available. was supposed to be fixed. However, this is not the case, but gpu works fine inside the container.

2. Steps to reproduce the issue

➜ docker run --gpus all --rm --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/tensorflow:20.03-tf2-py3 nvidia-smi

================
== TensorFlow ==
================

NVIDIA Release 20.03-tf2 (build 11026100)
TensorFlow Version 2.1.0

Container image Copyright (c) 2019, NVIDIA CORPORATION.  All rights reserved.
Copyright 2017-2019 The TensorFlow Authors.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION.  All rights reserved.
NVIDIA modifications are covered by the license terms that apply to the underlying project or file.

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
   Use 'nvidia-docker run' to start this container; see
   https://github.com/NVIDIA/nvidia-docker/wiki/nvidia-docker .

NOTE: MOFED driver for multi-node communication was not detected.
      Multi-node communication performance may be reduced.

Fri Sep 17 09:06:03 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.00       Driver Version: 510.06       CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
| N/A   38C    P8     3W /  N/A |    901MiB /  6144MiB |     N/A      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

3. Information to attach (optional if deemed irrelevant)

  • Some nvidia-container information: nvidia-container-cli -k -d /dev/tty info
➜ nvidia-container-cli -k -d /dev/tty info

-- WARNING, the following logs are for debugging purposes only --

I0917 09:07:42.181412 2113 nvc.c:372] initializing library context (version=1.5.1~rc.1, build=63fc23a302b462802a88b55b4437430d330743b2)
I0917 09:07:42.181502 2113 nvc.c:346] using root /
I0917 09:07:42.181509 2113 nvc.c:347] using ldcache /etc/ld.so.cache
I0917 09:07:42.181517 2113 nvc.c:348] using unprivileged user 1000:1000
I0917 09:07:42.181556 2113 nvc.c:389] attempting to load dxcore to see if we are running under Windows Subsystem for Linux (WSL)
I0917 09:07:42.229284 2113 dxcore.c:227] Creating a new WDDM Adapter for hAdapter:40000000 luid:1f4684e
I0917 09:07:42.261717 2113 dxcore.c:268] Adding new adapter via dxcore hAdapter:40000000 luid:1f4684e wddm version:3000
I0917 09:07:42.261786 2113 dxcore.c:227] Creating a new WDDM Adapter for hAdapter:40000040 luid:1f468a5
I0917 09:07:42.265534 2113 dxcore.c:210] Core Nvidia component libcuda.so.1.1 not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:07:42.267131 2113 dxcore.c:210] Core Nvidia component libcuda_loader.so not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:07:42.269144 2113 dxcore.c:210] Core Nvidia component libnvidia-ptxjitcompiler.so.1 not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:07:42.271089 2113 dxcore.c:210] Core Nvidia component libnvidia-ml.so.1 not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:07:42.273155 2113 dxcore.c:210] Core Nvidia component libnvidia-ml_loader.so not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:07:42.275406 2113 dxcore.c:210] Core Nvidia component nvidia-smi not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:07:42.275479 2113 dxcore.c:215] No Nvidia component found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
E0917 09:07:42.275484 2113 dxcore.c:261] Failed to query the core Nvidia libraries for the adapter. Skipping it.
I0917 09:07:42.275489 2113 dxcore.c:326] dxcore layer initialized successfully
W0917 09:07:42.276602 2113 nvc.c:397] skipping kernel modules load on WSL
I0917 09:07:42.277173 2114 driver.c:101] starting driver service
I0917 09:07:42.384389 2113 nvc_info.c:758] requesting driver information with ''
I0917 09:07:42.581586 2113 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvidia-opticalflow.so.1
I0917 09:07:42.584072 2113 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvidia-ml.so.1
I0917 09:07:42.586255 2113 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvidia-encode.so.1
I0917 09:07:42.588120 2113 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvcuvid.so.1
I0917 09:07:42.588275 2113 nvc_info.c:197] selecting /usr/lib/wsl/lib/libdxcore.so
I0917 09:07:42.588354 2113 nvc_info.c:197] selecting /usr/lib/wsl/lib/libcuda.so.1
W0917 09:07:42.588493 2113 nvc_info.c:397] missing library libnvidia-cfg.so
W0917 09:07:42.588540 2113 nvc_info.c:397] missing library libnvidia-nscq.so
W0917 09:07:42.588576 2113 nvc_info.c:397] missing library libnvidia-opencl.so
W0917 09:07:42.588585 2113 nvc_info.c:397] missing library libnvidia-ptxjitcompiler.so
W0917 09:07:42.588591 2113 nvc_info.c:397] missing library libnvidia-fatbinaryloader.so
W0917 09:07:42.588596 2113 nvc_info.c:397] missing library libnvidia-allocator.so
W0917 09:07:42.588600 2113 nvc_info.c:397] missing library libnvidia-compiler.so
W0917 09:07:42.588609 2113 nvc_info.c:397] missing library libnvidia-ngx.so
W0917 09:07:42.588612 2113 nvc_info.c:397] missing library libvdpau_nvidia.so
W0917 09:07:42.588616 2113 nvc_info.c:397] missing library libnvidia-eglcore.so
W0917 09:07:42.588620 2113 nvc_info.c:397] missing library libnvidia-glcore.so
W0917 09:07:42.588633 2113 nvc_info.c:397] missing library libnvidia-tls.so
W0917 09:07:42.588640 2113 nvc_info.c:397] missing library libnvidia-glsi.so
W0917 09:07:42.588644 2113 nvc_info.c:397] missing library libnvidia-fbc.so
W0917 09:07:42.588648 2113 nvc_info.c:397] missing library libnvidia-ifr.so
W0917 09:07:42.588689 2113 nvc_info.c:397] missing library libnvidia-rtcore.so
W0917 09:07:42.588733 2113 nvc_info.c:397] missing library libnvoptix.so
W0917 09:07:42.588741 2113 nvc_info.c:397] missing library libGLX_nvidia.so
W0917 09:07:42.588746 2113 nvc_info.c:397] missing library libEGL_nvidia.so
W0917 09:07:42.588751 2113 nvc_info.c:397] missing library libGLESv2_nvidia.so
W0917 09:07:42.588784 2113 nvc_info.c:397] missing library libGLESv1_CM_nvidia.so
W0917 09:07:42.588793 2113 nvc_info.c:397] missing library libnvidia-glvkspirv.so
W0917 09:07:42.588801 2113 nvc_info.c:397] missing library libnvidia-cbl.so
W0917 09:07:42.588805 2113 nvc_info.c:401] missing compat32 library libnvidia-ml.so
W0917 09:07:42.588814 2113 nvc_info.c:401] missing compat32 library libnvidia-cfg.so
W0917 09:07:42.588820 2113 nvc_info.c:401] missing compat32 library libnvidia-nscq.so
W0917 09:07:42.588825 2113 nvc_info.c:401] missing compat32 library libcuda.so
W0917 09:07:42.588830 2113 nvc_info.c:401] missing compat32 library libnvidia-opencl.so
W0917 09:07:42.588845 2113 nvc_info.c:401] missing compat32 library libnvidia-ptxjitcompiler.so
W0917 09:07:42.588852 2113 nvc_info.c:401] missing compat32 library libnvidia-fatbinaryloader.so
W0917 09:07:42.588883 2113 nvc_info.c:401] missing compat32 library libnvidia-allocator.so
W0917 09:07:42.588894 2113 nvc_info.c:401] missing compat32 library libnvidia-compiler.so
W0917 09:07:42.588899 2113 nvc_info.c:401] missing compat32 library libnvidia-ngx.so
W0917 09:07:42.588905 2113 nvc_info.c:401] missing compat32 library libvdpau_nvidia.so
W0917 09:07:42.588914 2113 nvc_info.c:401] missing compat32 library libnvidia-encode.so
W0917 09:07:42.588920 2113 nvc_info.c:401] missing compat32 library libnvidia-opticalflow.so
W0917 09:07:42.588925 2113 nvc_info.c:401] missing compat32 library libnvcuvid.so
W0917 09:07:42.588936 2113 nvc_info.c:401] missing compat32 library libnvidia-eglcore.so
W0917 09:07:42.589011 2113 nvc_info.c:401] missing compat32 library libnvidia-glcore.so
W0917 09:07:42.589020 2113 nvc_info.c:401] missing compat32 library libnvidia-tls.so
W0917 09:07:42.589029 2113 nvc_info.c:401] missing compat32 library libnvidia-glsi.so
W0917 09:07:42.589035 2113 nvc_info.c:401] missing compat32 library libnvidia-fbc.so
W0917 09:07:42.589068 2113 nvc_info.c:401] missing compat32 library libnvidia-ifr.so
W0917 09:07:42.589075 2113 nvc_info.c:401] missing compat32 library libnvidia-rtcore.so
W0917 09:07:42.589084 2113 nvc_info.c:401] missing compat32 library libnvoptix.so
W0917 09:07:42.589089 2113 nvc_info.c:401] missing compat32 library libGLX_nvidia.so
W0917 09:07:42.589098 2113 nvc_info.c:401] missing compat32 library libEGL_nvidia.so
W0917 09:07:42.589104 2113 nvc_info.c:401] missing compat32 library libGLESv2_nvidia.so
W0917 09:07:42.589109 2113 nvc_info.c:401] missing compat32 library libGLESv1_CM_nvidia.so
W0917 09:07:42.589115 2113 nvc_info.c:401] missing compat32 library libnvidia-glvkspirv.so
W0917 09:07:42.589120 2113 nvc_info.c:401] missing compat32 library libnvidia-cbl.so
W0917 09:07:42.589123 2113 nvc_info.c:401] missing compat32 library libdxcore.so
I0917 09:07:42.592472 2113 nvc_info.c:277] selecting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/nvidia-smi
W0917 09:07:43.181748 2113 nvc_info.c:423] missing binary nvidia-debugdump
W0917 09:07:43.181804 2113 nvc_info.c:423] missing binary nvidia-persistenced
W0917 09:07:43.181811 2113 nvc_info.c:423] missing binary nv-fabricmanager
W0917 09:07:43.181815 2113 nvc_info.c:423] missing binary nvidia-cuda-mps-control
W0917 09:07:43.181858 2113 nvc_info.c:423] missing binary nvidia-cuda-mps-server
I0917 09:07:43.181864 2113 nvc_info.c:437] skipping path lookup for dxcore
I0917 09:07:43.181878 2113 nvc_info.c:520] listing device /dev/dxg
W0917 09:07:43.182044 2113 nvc_info.c:347] missing ipc path /var/run/nvidia-persistenced/socket
W0917 09:07:43.182084 2113 nvc_info.c:347] missing ipc path /var/run/nvidia-fabricmanager/socket
W0917 09:07:43.182147 2113 nvc_info.c:347] missing ipc path /tmp/nvidia-mps
I0917 09:07:43.182190 2113 nvc_info.c:814] requesting device information with ''
I0917 09:07:43.203401 2113 nvc_info.c:686] listing dxcore adapter 0 (GPU-d8374a7e-bb23-a73e-bb4e-5421ad9ea115 at 00000000:01:00.0)
NVRM version:   510.06
CUDA version:   11.6

Device Index:   0
Device Minor:   0
Model:          NVIDIA GeForce GTX 1660 Ti with Max-Q Design
Brand:          GeForce
GPU UUID:       GPU-d8374a7e-bb23-a73e-bb4e-5421ad9ea115
Bus Location:   00000000:01:00.0
Architecture:   7.5
I0917 09:07:43.203478 2113 nvc.c:423] shutting down library context
I0917 09:07:43.205610 2114 driver.c:163] terminating driver service
I0917 09:07:43.210303 2113 driver.c:203] driver service terminated successfully
  • Kernel version from uname -a
    Linux DESKTOP-BRNIA5E 5.10.43.3-microsoft-standard-WSL2 #1 SMP Wed Jun 16 23:47:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
  • Any relevant kernel output lines from dmesg
  • Driver information from nvidia-smi -a
➜ nvidia-smi -a

==============NVSMI LOG==============

Timestamp                                 : Fri Sep 17 11:09:53 2021
Driver Version                            : 510.06
CUDA Version                              : 11.6

Attached GPUs                             : 1
GPU 00000000:01:00.0
   Product Name                          : NVIDIA GeForce GTX 1660 Ti with Max-Q Design
   Product Brand                         : GeForce
   Product Architecture                  : Turing
   Display Mode                          : Disabled
   Display Active                        : Disabled
   Persistence Mode                      : Enabled
   MIG Mode
       Current                           : N/A
       Pending                           : N/A
   Accounting Mode                       : Disabled
   Accounting Mode Buffer Size           : 4000
   Driver Model
       Current                           : WDDM
       Pending                           : WDDM
   Serial Number                         : N/A
   GPU UUID                              : GPU-d8374a7e-bb23-a73e-bb4e-5421ad9ea115
   Minor Number                          : N/A
   VBIOS Version                         : 90.16.4c.00.38
   MultiGPU Board                        : No
   Board ID                              : 0x100
   GPU Part Number                       : N/A
   Module ID                             : 0
   Inforom Version
       Image Version                     : G001.0000.02.04
       OEM Object                        : 1.1
       ECC Object                        : N/A
       Power Management Object           : N/A
   GPU Operation Mode
       Current                           : N/A
       Pending                           : N/A
   GSP Firmware Version                  : N/A
   GPU Virtualization Mode
       Virtualization Mode               : None
       Host VGPU Mode                    : N/A
   IBMNPU
       Relaxed Ordering Mode             : N/A
   PCI
       Bus                               : 0x01
       Device                            : 0x00
       Domain                            : 0x0000
       Device Id                         : 0x219110DE
       Bus Id                            : 00000000:01:00.0
       Sub System Id                     : 0x17EF1043
       GPU Link Info
           PCIe Generation
               Max                       : 3
               Current                   : 3
           Link Width
               Max                       : 16x
               Current                   : 8x
       Bridge Chip
           Type                          : N/A
           Firmware                      : N/A
       Replays Since Reset               : 0
       Replay Number Rollovers           : 0
       Tx Throughput                     : 716000 KB/s
       Rx Throughput                     : 6000 KB/s
   Fan Speed                             : N/A
   Performance State                     : P8
   Clocks Throttle Reasons
       Idle                              : Active
       Applications Clocks Setting       : Not Active
       SW Power Cap                      : Not Active
       HW Slowdown                       : Not Active
           HW Thermal Slowdown           : Not Active
           HW Power Brake Slowdown       : Not Active
       Sync Boost                        : Not Active
       SW Thermal Slowdown               : Not Active
       Display Clock Setting             : Not Active
   FB Memory Usage
       Total                             : 6144 MiB
       Used                              : 896 MiB
       Free                              : 5248 MiB
   BAR1 Memory Usage
       Total                             : 256 MiB
       Used                              : 2 MiB
       Free                              : 254 MiB
   Compute Mode                          : Default
   Utilization
       Gpu                               : N/A
       Memory                            : N/A
       Encoder                           : 0 %
       Decoder                           : 0 %
   Encoder Stats
       Active Sessions                   : 0
       Average FPS                       : 0
       Average Latency                   : 0
   FBC Stats
       Active Sessions                   : 0
       Average FPS                       : 0
       Average Latency                   : 0
   Ecc Mode
       Current                           : N/A
       Pending                           : N/A
   ECC Errors
       Volatile
           SRAM Correctable              : N/A
           SRAM Uncorrectable            : N/A
           DRAM Correctable              : N/A
           DRAM Uncorrectable            : N/A
       Aggregate
           SRAM Correctable              : N/A
           SRAM Uncorrectable            : N/A
           DRAM Correctable              : N/A
           DRAM Uncorrectable            : N/A
   Retired Pages
       Single Bit ECC                    : N/A
       Double Bit ECC                    : N/A
       Pending Page Blacklist            : N/A
   Remapped Rows                         : N/A
   Temperature
       GPU Current Temp                  : 38 C
       GPU Shutdown Temp                 : 100 C
       GPU Slowdown Temp                 : 95 C
       GPU Max Operating Temp            : 87 C
       GPU Target Temperature            : N/A
       Memory Current Temp               : N/A
       Memory Max Operating Temp         : N/A
   Power Readings
       Power Management                  : N/A
       Power Draw                        : 3.81 W
       Power Limit                       : N/A
       Default Power Limit               : N/A
       Enforced Power Limit              : N/A
       Min Power Limit                   : N/A
       Max Power Limit                   : N/A
   Clocks
       Graphics                          : 22 MHz
       SM                                : 22 MHz
       Memory                            : 58 MHz
       Video                             : 539 MHz
   Applications Clocks
       Graphics                          : N/A
       Memory                            : N/A
   Default Applications Clocks
       Graphics                          : N/A
       Memory                            : N/A
   Max Clocks
       Graphics                          : 2100 MHz
       SM                                : 2100 MHz
       Memory                            : 6001 MHz
       Video                             : 1950 MHz
   Max Customer Boost Clocks
       Graphics                          : N/A
   Clock Policy
       Auto Boost                        : N/A
       Auto Boost Default                : N/A
   Voltage
       Graphics                          : N/A
   Processes                             : None
  • Docker version from docker version
Client: Docker Engine - Community
Version:           20.10.8
API version:       1.41
Go version:        go1.16.6
Git commit:        3967b7d
Built:             Fri Jul 30 19:54:27 2021
OS/Arch:           linux/amd64
Context:           default
Experimental:      true

Server: Docker Engine - Community
Engine:
 Version:          20.10.8
 API version:      1.41 (minimum version 1.12)
 Go version:       go1.16.6
 Git commit:       75249d8
 Built:            Fri Jul 30 19:52:33 2021
 OS/Arch:          linux/amd64
 Experimental:     false
containerd:
 Version:          1.4.9
 GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
runc:
 Version:          1.0.1
 GitCommit:        v1.0.1-0-g4144b63
docker-init:
 Version:          0.19.0
 GitCommit:        de40ad0
  • NVIDIA packages version from dpkg -l '*nvidia*' or rpm -qa '*nvidia*'
un  libgldispatch0-nvidia         <none>       <none>       (no description available)
ii  libnvidia-container-tools     1.5.1~rc.1-1 amd64        NVIDIA container runtime library (command-line tools)
ii  libnvidia-container1:amd64    1.5.1~rc.1-1 amd64        NVIDIA container runtime library
ii  nvidia-container-runtime      3.5.0-1      amd64        NVIDIA container runtime
un  nvidia-container-runtime-hook <none>       <none>       (no description available)
ii  nvidia-container-toolkit      1.5.1-1      amd64        NVIDIA container runtime hook
un  nvidia-docker                 <none>       <none>       (no description available)
ii  nvidia-docker2                2.6.0-1      all          nvidia-docker CLI wrapper
  • NVIDIA container library version from nvidia-container-cli -V
version: 1.5.1~rc.1
build date: 2021-09-15T08:55+00:00
build revision: 63fc23a302b462802a88b55b4437430d330743b2
build compiler: x86_64-linux-gnu-gcc-7 7.5.0
build platform: x86_64
build flags: -D_GNU_SOURCE -D_FORTIFY_SOURCE=2 -DNDEBUG -std=gnu11 -O2 -g -fdata-sections -ffunction-sections -fstack-protector -fno-strict-aliasing -fvisibility=hidden -Wall -Wextra -Wcast-align -Wpointer-arith -Wmissing-prototypes -Wnonnull -Wwrite-strings -Wlogical-op -Wformat=2 -Wmissing-format-attribute -Winit-self -Wshadow -Wstrict-prototypes -Wunreachable-code -Wconversion -Wsign-conversion -Wno-unknown-warning-option -Wno-format-extra-args -Wno-gnu-alignof-expression -Wl,-zrelro -Wl,-znow -Wl,-zdefs -Wl,--gc-sections
-- WARNING, the following logs are for debugging purposes only --

I0917 09:13:37.025674 2466 nvc.c:372] initializing library context (version=1.5.1~rc.1, build=63fc23a302b462802a88b55b4437430d330743b2)
I0917 09:13:37.025752 2466 nvc.c:346] using root /
I0917 09:13:37.025764 2466 nvc.c:347] using ldcache /etc/ld.so.cache
I0917 09:13:37.025775 2466 nvc.c:348] using unprivileged user 65534:65534
I0917 09:13:37.025810 2466 nvc.c:389] attempting to load dxcore to see if we are running under Windows Subsystem for Linux (WSL)
I0917 09:13:37.086002 2466 dxcore.c:227] Creating a new WDDM Adapter for hAdapter:40000000 luid:1f4684e
I0917 09:13:37.129039 2466 dxcore.c:268] Adding new adapter via dxcore hAdapter:40000000 luid:1f4684e wddm version:3000
I0917 09:13:37.129117 2466 dxcore.c:227] Creating a new WDDM Adapter for hAdapter:40000040 luid:1f468a5
I0917 09:13:37.133523 2466 dxcore.c:210] Core Nvidia component libcuda.so.1.1 not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:13:37.135694 2466 dxcore.c:210] Core Nvidia component libcuda_loader.so not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:13:37.137779 2466 dxcore.c:210] Core Nvidia component libnvidia-ptxjitcompiler.so.1 not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:13:37.139831 2466 dxcore.c:210] Core Nvidia component libnvidia-ml.so.1 not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:13:37.142211 2466 dxcore.c:210] Core Nvidia component libnvidia-ml_loader.so not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:13:37.144242 2466 dxcore.c:210] Core Nvidia component nvidia-smi not found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
I0917 09:13:37.144285 2466 dxcore.c:215] No Nvidia component found in /usr/lib/wsl/drivers/u0371177.inf_amd64_1e6eae2400657e62
E0917 09:13:37.144298 2466 dxcore.c:261] Failed to query the core Nvidia libraries for the adapter. Skipping it.
I0917 09:13:37.144311 2466 dxcore.c:326] dxcore layer initialized successfully
W0917 09:13:37.145358 2466 nvc.c:397] skipping kernel modules load on WSL
I0917 09:13:37.145848 2473 driver.c:101] starting driver service
I0917 09:13:37.330223 2466 nvc_container.c:388] configuring container with 'compute utility video supervised'
I0917 09:13:37.330927 2466 nvc_container.c:408] setting pid to 2459
I0917 09:13:37.330961 2466 nvc_container.c:409] setting rootfs to /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged
I0917 09:13:37.330970 2466 nvc_container.c:410] setting owner to 0:0
I0917 09:13:37.330979 2466 nvc_container.c:411] setting bins directory to /usr/bin
I0917 09:13:37.330987 2466 nvc_container.c:412] setting libs directory to /usr/lib/x86_64-linux-gnu
I0917 09:13:37.330996 2466 nvc_container.c:413] setting libs32 directory to /usr/lib/i386-linux-gnu
I0917 09:13:37.331004 2466 nvc_container.c:414] setting cudart directory to /usr/local/cuda
I0917 09:13:37.331012 2466 nvc_container.c:415] setting ldconfig to @/sbin/ldconfig.real (host relative)
I0917 09:13:37.331021 2466 nvc_container.c:416] setting mount namespace to /proc/2459/ns/mnt
I0917 09:13:37.331029 2466 nvc_container.c:418] setting devices cgroup to /sys/fs/cgroup/devices/docker/d35f4e16ed3e0b61a5582369e306a14c55a2fee0af994de4331102bcd3c9b034
I0917 09:13:37.331041 2466 nvc_info.c:758] requesting driver information with ''
I0917 09:13:37.604554 2466 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvidia-opticalflow.so.1
I0917 09:13:37.606579 2466 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvidia-ml.so.1
I0917 09:13:37.608506 2466 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvidia-encode.so.1
I0917 09:13:37.610439 2466 nvc_info.c:197] selecting /usr/lib/wsl/lib/libnvcuvid.so.1
I0917 09:13:37.610560 2466 nvc_info.c:197] selecting /usr/lib/wsl/lib/libdxcore.so
I0917 09:13:37.610596 2466 nvc_info.c:197] selecting /usr/lib/wsl/lib/libcuda.so.1
W0917 09:13:37.610663 2466 nvc_info.c:397] missing library libnvidia-cfg.so
W0917 09:13:37.610669 2466 nvc_info.c:397] missing library libnvidia-nscq.so
W0917 09:13:37.610674 2466 nvc_info.c:397] missing library libnvidia-opencl.so
W0917 09:13:37.610678 2466 nvc_info.c:397] missing library libnvidia-ptxjitcompiler.so
W0917 09:13:37.610682 2466 nvc_info.c:397] missing library libnvidia-fatbinaryloader.so
W0917 09:13:37.610687 2466 nvc_info.c:397] missing library libnvidia-allocator.so
W0917 09:13:37.610691 2466 nvc_info.c:397] missing library libnvidia-compiler.so
W0917 09:13:37.610696 2466 nvc_info.c:397] missing library libnvidia-ngx.so
W0917 09:13:37.610700 2466 nvc_info.c:397] missing library libvdpau_nvidia.so
W0917 09:13:37.610704 2466 nvc_info.c:397] missing library libnvidia-eglcore.so
W0917 09:13:37.610709 2466 nvc_info.c:397] missing library libnvidia-glcore.so
W0917 09:13:37.610713 2466 nvc_info.c:397] missing library libnvidia-tls.so
W0917 09:13:37.610717 2466 nvc_info.c:397] missing library libnvidia-glsi.so
W0917 09:13:37.610722 2466 nvc_info.c:397] missing library libnvidia-fbc.so
W0917 09:13:37.610726 2466 nvc_info.c:397] missing library libnvidia-ifr.so
W0917 09:13:37.610731 2466 nvc_info.c:397] missing library libnvidia-rtcore.so
W0917 09:13:37.610735 2466 nvc_info.c:397] missing library libnvoptix.so
W0917 09:13:37.610739 2466 nvc_info.c:397] missing library libGLX_nvidia.so
W0917 09:13:37.610744 2466 nvc_info.c:397] missing library libEGL_nvidia.so
W0917 09:13:37.610748 2466 nvc_info.c:397] missing library libGLESv2_nvidia.so
W0917 09:13:37.610752 2466 nvc_info.c:397] missing library libGLESv1_CM_nvidia.so
W0917 09:13:37.610757 2466 nvc_info.c:397] missing library libnvidia-glvkspirv.so
W0917 09:13:37.610761 2466 nvc_info.c:397] missing library libnvidia-cbl.so
W0917 09:13:37.610766 2466 nvc_info.c:401] missing compat32 library libnvidia-ml.so
W0917 09:13:37.610770 2466 nvc_info.c:401] missing compat32 library libnvidia-cfg.so
W0917 09:13:37.610774 2466 nvc_info.c:401] missing compat32 library libnvidia-nscq.so
W0917 09:13:37.610779 2466 nvc_info.c:401] missing compat32 library libcuda.so
W0917 09:13:37.610783 2466 nvc_info.c:401] missing compat32 library libnvidia-opencl.so
W0917 09:13:37.610787 2466 nvc_info.c:401] missing compat32 library libnvidia-ptxjitcompiler.so
W0917 09:13:37.610792 2466 nvc_info.c:401] missing compat32 library libnvidia-fatbinaryloader.so
W0917 09:13:37.610796 2466 nvc_info.c:401] missing compat32 library libnvidia-allocator.so
W0917 09:13:37.610814 2466 nvc_info.c:401] missing compat32 library libnvidia-compiler.so
W0917 09:13:37.610818 2466 nvc_info.c:401] missing compat32 library libnvidia-ngx.so
W0917 09:13:37.610823 2466 nvc_info.c:401] missing compat32 library libvdpau_nvidia.so
W0917 09:13:37.610827 2466 nvc_info.c:401] missing compat32 library libnvidia-encode.so
W0917 09:13:37.610832 2466 nvc_info.c:401] missing compat32 library libnvidia-opticalflow.so
W0917 09:13:37.610836 2466 nvc_info.c:401] missing compat32 library libnvcuvid.so
W0917 09:13:37.610840 2466 nvc_info.c:401] missing compat32 library libnvidia-eglcore.so
W0917 09:13:37.610845 2466 nvc_info.c:401] missing compat32 library libnvidia-glcore.so
W0917 09:13:37.610849 2466 nvc_info.c:401] missing compat32 library libnvidia-tls.so
W0917 09:13:37.610853 2466 nvc_info.c:401] missing compat32 library libnvidia-glsi.so
W0917 09:13:37.610858 2466 nvc_info.c:401] missing compat32 library libnvidia-fbc.so
W0917 09:13:37.610862 2466 nvc_info.c:401] missing compat32 library libnvidia-ifr.so
W0917 09:13:37.610866 2466 nvc_info.c:401] missing compat32 library libnvidia-rtcore.so
W0917 09:13:37.610871 2466 nvc_info.c:401] missing compat32 library libnvoptix.so
W0917 09:13:37.610875 2466 nvc_info.c:401] missing compat32 library libGLX_nvidia.so
W0917 09:13:37.610879 2466 nvc_info.c:401] missing compat32 library libEGL_nvidia.so
W0917 09:13:37.610883 2466 nvc_info.c:401] missing compat32 library libGLESv2_nvidia.so
W0917 09:13:37.610888 2466 nvc_info.c:401] missing compat32 library libGLESv1_CM_nvidia.so
W0917 09:13:37.610892 2466 nvc_info.c:401] missing compat32 library libnvidia-glvkspirv.so
W0917 09:13:37.610896 2466 nvc_info.c:401] missing compat32 library libnvidia-cbl.so
W0917 09:13:37.610901 2466 nvc_info.c:401] missing compat32 library libdxcore.so
I0917 09:13:37.614770 2466 nvc_info.c:277] selecting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/nvidia-smi
W0917 09:13:37.619030 2466 nvc_info.c:423] missing binary nvidia-debugdump
W0917 09:13:37.619057 2466 nvc_info.c:423] missing binary nvidia-persistenced
W0917 09:13:37.619062 2466 nvc_info.c:423] missing binary nv-fabricmanager
W0917 09:13:37.619066 2466 nvc_info.c:423] missing binary nvidia-cuda-mps-control
W0917 09:13:37.619071 2466 nvc_info.c:423] missing binary nvidia-cuda-mps-server
I0917 09:13:37.619075 2466 nvc_info.c:437] skipping path lookup for dxcore
I0917 09:13:37.619085 2466 nvc_info.c:520] listing device /dev/dxg
W0917 09:13:37.619105 2466 nvc_info.c:347] missing ipc path /var/run/nvidia-persistenced/socket
W0917 09:13:37.619120 2466 nvc_info.c:347] missing ipc path /var/run/nvidia-fabricmanager/socket
W0917 09:13:37.619133 2466 nvc_info.c:347] missing ipc path /tmp/nvidia-mps
I0917 09:13:37.619139 2466 nvc_info.c:814] requesting device information with ''
I0917 09:13:37.638612 2466 nvc_info.c:686] listing dxcore adapter 0 (GPU-d8374a7e-bb23-a73e-bb4e-5421ad9ea115 at 00000000:01:00.0)
W0917 09:13:37.638731 2466 nvc_mount.c:730] skipping procfs mount on WSL
I0917 09:13:37.640957 2466 nvc_mount.c:112] mounting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/nvidia-smi at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/bin/nvidia-smi
I0917 09:13:37.643833 2466 nvc_mount.c:112] mounting /usr/lib/wsl/lib/libnvidia-ml.so.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1
I0917 09:13:37.646402 2466 nvc_mount.c:112] mounting /usr/lib/wsl/lib/libcuda.so.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libcuda.so.1
I0917 09:13:37.648233 2466 nvc_mount.c:112] mounting /usr/lib/wsl/lib/libnvidia-encode.so.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libnvidia-encode.so.1
I0917 09:13:37.650949 2466 nvc_mount.c:112] mounting /usr/lib/wsl/lib/libnvidia-opticalflow.so.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.1
I0917 09:13:37.653734 2466 nvc_mount.c:112] mounting /usr/lib/wsl/lib/libnvcuvid.so.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libnvcuvid.so.1
I0917 09:13:37.655893 2466 nvc_mount.c:112] mounting /usr/lib/wsl/lib/libdxcore.so at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libdxcore.so
I0917 09:13:37.655936 2466 nvc_mount.c:524] creating symlink /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libcuda.so -> libcuda.so.1
I0917 09:13:37.655981 2466 nvc_mount.c:524] creating symlink /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so -> libnvidia-opticalflow.so.1
I0917 09:13:37.656037 2466 nvc_mount.c:208] mounting /dev/dxg at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/dev/dxg
I0917 09:13:37.656069 2466 nvc_mount.c:499] whitelisting device node 10:63
I0917 09:13:37.656465 2466 nvc_mount.c:163] mounting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libcuda.so.1.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libcuda.so.1.1
I0917 09:13:37.656503 2466 nvc_mount.c:163] mounting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libcuda_loader.so at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libcuda_loader.so
I0917 09:13:37.658065 2466 nvc_mount.c:163] mounting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libnvidia-ptxjitcompiler.so.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libnvidia-ptxjitcompiler.so.1
I0917 09:13:37.660070 2466 nvc_mount.c:163] mounting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libnvidia-ml.so.1 at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libnvidia-ml.so.1
I0917 09:13:37.660165 2466 nvc_mount.c:163] mounting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libnvidia-ml_loader.so at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/libnvidia-ml_loader.so
I0917 09:13:37.661639 2466 nvc_mount.c:163] mounting /usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/nvidia-smi at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged/usr/lib/wsl/drivers/nvami.inf_amd64_8dec68894713a48d/nvidia-smi
I0917 09:13:37.661713 2466 nvc_ldcache.c:362] executing /sbin/ldconfig.real from host at /var/lib/docker/overlay2/9c666851f8d5bf1fbe2e444da05dd263105158ecc2ae33bb3f75be5fa2d7106a/merged
I0917 09:13:37.760403 2466 nvc.c:423] shutting down library context
I0917 09:13:37.762498 2473 driver.c:163] terminating driver service
I0917 09:13:37.764554 2466 driver.c:203] driver service terminated successfully
  • Docker command, image and tag used
docker run --gpus all --rm --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/tensorflow:20.03-tf2-py3 nvidia-smi

4. Additional information

OS: Windows 11 Dev Build 22449.1000 (from winver)
WSL2 distro: Ubuntu 20.04
@PQLLUX PQLLUX mentioned this issue Oct 5, 2021
Open
@SinanAkkoyun
Copy link

Same thing here

@SinanAkkoyun
Copy link

For me it worked with the newer version:
docker run --gpus all --rm --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/tensorflow:22.01-tf2-py3 nvidia-smi

@PQLLUX PQLLUX closed this as completed Apr 2, 2022
@shuifeng1988
Copy link

same problem, anyone solved it?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@shuifeng1988 @SinanAkkoyun @PQLLUX