The support for 3080 or 3090

[WangWenhao0716]

Hi! I have got several 3080 GPUs. However, I find some problems.
The system is Ubuntu 16.04, the version of PyTorch is 1.1.0.
My codes are:
import torch ---(1)
X = torch.rand((3,6)) ---(2)
X = X.cuda() ---(3)

I find the step (1) and (2) very smooth, which means it nearly does not cost any time.
However, I found the third step costs nearly 10 minutes to finish moving the data from memory to VRAM.

I do not know what brings the problem and I also tried other versions of GPU and other 3080s.

I'm looking forward to your reply. Thanks!

cc @malfet @seemethere @walterddr @ngimel

[xwang233]

You are using pytorch 1.1, which is not compiled (optimized) for cuda compute capability 8.x (Ampere). Please get the latest CUDA 11. For better performance, please also get the latest pytorch source code, and build them with TORCH_CUDA_ARCH_LIST=8.6.

You can also try a cuda 11 pytorch nightly build here https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html.

Please try the latest pytorch wheel built with cuda 11.x here https://pytorch.org/get-started/locally/

[WangWenhao0716]

You are using pytorch 1.1, which is not compiled (optimized) for cuda compute capability 8.x (Ampere). Please get the latest CUDA 11. For better performance, please also get the latest pytorch source code, and build them with TORCH_CUDA_ARCH_LIST=8.6.

Thanks for your reply. I have used your method. However, the problem is
nvcc fatal : Unsupported gpu architecture 'compute_86'
I do not know how to do it?

[xwang233]

Seems like the latest cuda 11 doesn't support sm_86 yet. Can you try TORCH_CUDA_ARCH_LIST='8.0+PTX'?

[WangWenhao0716]

Seems like the latest cuda 11 doesn't support sm_86 yet. Can you try TORCH_CUDA_ARCH_LIST='8.0+PTX'?

Thanks, I have tried TORCH_CUDA_ARCH_LIST='8.0+PTX'. The problem "nvcc fatal : Unsupported gpu architecture 'compute_86'" is gone. And I compile it successfully. By the way, " a cuda 11 pytorch nightly" also works well.

However, a warning indicates:
GeForce RTX 3080 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_80 compute_80.

[WangWenhao0716]

I'm wondering when and whether the pytorch/Nvidia official can give support to sm_86?

[xwang233]

Thanks. Are you able to train network on that install? If not, then we may have to wait for the next cuda release that supports sm_86.

[WangWenhao0716]

Thanks. Are you able to train network on that install? If not, then we may have to wait for the next cuda release that supports sm_86.

I will try it and give your feedback later.

[WangWenhao0716]

Thanks. Are you able to train network on that install? If not, then we may have to wait for the next cuda release that supports sm_86.

Currently, I try:
import torch
X = torch.rand((64,3,256,128)).cuda()
import torchvision
model = torchvision.models.resnet50()
model = model.cuda()
print(model(X))

It works well.

[WangWenhao0716]

@xwang233
The Deep Learning Performance Test for 3080
Test environment:
One 3080 GPU (provided by Colorful), PyTorch ‘1.7.0a0+9e5045e’,
One 1080Ti GPU (Founder Edition), PyTorch ‘1.1.0’
The PyTorch for 3080 is a beta version, and it is compiled by myself using CUDA 11.0. However, 3080 with CUDA capability sm_86 is not compatible with this version. Therefore, performance is limited.
Backbone: ResNet-50
Task: Person Re-identification
Test Results:
Speed: The speed of the beta version is 1.5 times as fast as 1080Ti. It is expected that the speed will be faster after the support of sm_86.
Temperature: The turbofan speed of 1080Ti is set to 90%, and violent fan ventilation is used. Its temperature is 74 Celsius. We do not add any restrictions on 3080, i.e. no fan ventilation, the GPU fan automatically starts and stops. The fan speed is 60% and the temperature is 70 Celsius.

[malfet]

@WangWenhao0716 can you try using PyTorch nightly builds (by running pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html ) and share the perf difference here?

[WangWenhao0716]

@malfet OK, in fact, I have downloaded the nightly version Pytorch and used it. The experiments show a significant improvement on multi-GPUs training because I do not compile Pytorch using NCCL by myself. I will report the detailed results soon.

[WangWenhao0716]

@malfet Currently, I'm waiting for the arrival of the last GPU. When it arrives, I will report the performance difference of 4 GPUs between 3080 and 1080Ti.

[MRzNone]

Will 3090 support memory pooling in pytorch?

[EricLe-dev]

I got the 3090 today. Cuda 11.1 + latest official Torch version does not work. I got "no kernel image is available for execution on the device" error.

[EricLe-dev]

Maybe the nightly build will work but I nuked the system entirely. Anyway, I will get the next 3090 in a few days and test if parallel training (multiple GPUs) work on this nightly version.

[bhchiang]

@WangWenhao0716 @xwang233 Do you know if it is safe now to compile with TORCH_CUDA_ARCH_LIST=8.6 with CUDA 11.1? Thanks!

[WangWenhao0716]

@bryanhpchiang Hi, now I'm thinking about this question!
First, I find the nightly versions of Pytorch, 1.7.0, and 1.8.0, are still based on 11.0. (When I install it, I find "cudatoolkit-11.0.221"). Therefore, I think the power of 3080/3090 is not released fully (My test on 3080/3090 using Pytorch convinced this point).
Second, I'm busying finishing my paper using servers, therefore, I have no time to "play" with 3080/3090.

Therefore,
@xwang233 If the first point I said is right? And when will PyTorch fully support 11.1?
@bryanhpchiang Could you try to compile PyTorch using TORCH_CUDA_ARCH_LIST=8.6 with CUDA 11.1? I think it is possible!
I'm looking forward to your compilation.

[xwang233]

It should be fine to build pytorch with TORCH_CUDA_ARCH_LIST=8.6 on cuda 11.1 from source.

I'm not familiar if pytorch release will use cuda 11.1 cc @malfet

[WangWenhao0716]

@xwang233 I tried to compile it by myself using cuda11.1. However, this time, I'm facing "fatal error: magma.h: No such file or directory"
I have installed magma-cuda11.0, but it does not help. And I cannot get magma-cuda11.1.
Do you have any solutions?

[WangWenhao0716]

@malfet Hi, I think currently Pytorch does not support cuda11.1. And the performance of 3080 and 3090 is limited. Is that right? When will Pytorch fully support cuda11.1?

[ngimel]

cc @ptrblck for 11.1 perf vs 11.0 perf for 3080

[luobin97]

3080 gpu can now use pytorch？

[WangWenhao0716]

Yes but it does not perform ideally. 发自我的 iPad

…

------------------ Original ------------------ From: luobin97 <notifications@github.com&gt; Date: Sat,Oct 10,2020 4:17 PM To: pytorch/pytorch <pytorch@noreply.github.com&gt; Cc: WenhaoWang <2609667791@qq.com&gt;, Mention <mention@noreply.github.com&gt; Subject: Re: [pytorch/pytorch] The support for 3080 or 3090 (#45021) 3080 gpu can now use pytorch？ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[Heermosi]

I'm working on cuda 11.1 and cudnn 8.04
I tried to specify sm_86 into the magma 2.5.4 compilation and it failed to recognize it! All right I used Ampere instead.
And when I tried to compile pytorch 1.6.0 stable version, it failed on caffe2 utils compilation, it was a not support gpu arch error.
So which version should I have to build and what kind of parameter should I provide with ./configure?
Thanks.

[Heermosi]

@xwang233 I tried to compile it by myself using cuda11.1. However, this time, I'm facing "fatal error: magma.h: No such file or directory"
I have installed magma-cuda11.0, but it does not help. And I cannot get magma-cuda11.1.
Do you have any solutions?

自己编，没办法

[wellhowtosay]

NO, just install it by:

conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch

but I still got :
eForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation

my cuda version is 11.1

[WangWenhao0716]

NO, just install it by:

conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch

but I still got :
eForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation

my cuda version is 11.1

I do not think it is possible because it works on my 3090. Please uninstall PyTorch thoroughly and reinstall it. You must point out cudatoolkit=11.0.

[wellhowtosay]

NO, just install it by:

conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch

but I still got :
eForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation
my cuda version is 11.1

I do not think it is possible because it works on my 3090. Please uninstall PyTorch thoroughly and reinstall it. You must point out cudatoolkit=11.0.

you're right ,I just somehow installed torch1.7 with pip . problem solved ! thank you! 兄弟你太牛逼了！

[WangWenhao0716]

hhhhh

[wellhowtosay]

hhhhh

sorry but, it failed again with: RuntimeError: nvrtc: error: invalid value for --gpu-architecture (-arch)
you sure the command works fine with cuda11.1? or maybe the right command is
conda install pytorch torchvision cudatoolkit=11 -c **pytorch-nightly**
which you mentioned above?

[WangWenhao0716]

please add my wechat&nbsp; Perhaps we can discuss this problem further My wechat 18602459215 发自我的&nbsp;iPad

…

------------------ Original ------------------ From: wellhowtosay <notifications@github.com&gt; Date: Wed,Nov 4,2020 0:21 AM To: pytorch/pytorch <pytorch@noreply.github.com&gt; Cc: Wenhao Wang <2609667791@qq.com&gt;, Mention <mention@noreply.github.com&gt; Subject: Re: [pytorch/pytorch] The support for 3080 or 3090 (#45021) hhhhh sorry but, it failed again with: RuntimeError: nvrtc: error: invalid value for --gpu-architecture (-arch) you sure the command works fine with cuda11.1? or maybe the right command is conda install pytorch torchvision cudatoolkit=11 -c **pytorch-nightly** which you mentioned above? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[johndpope]

When I ran some code for pytorch it says to goto https://pytorch.org/get-started/locally/ - but there's no mention of 3090 on the screen - so I ended up here.

➜ stylegan2-pytorch git:(master) python convert_weight.py --repo ../stylegan2 stylegan2-ffhq-config-f.pkl
/home/jp/miniconda3/envs/tf2/lib/python3.8/site-packages/torch/cuda/init.py:104: UserWarning:
GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70 sm_75.
If you want to use the GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

UPDATE / just grab the latest nightly build
https://discuss.pytorch.org/t/rtx-3000-support/98158/5

https://pytorch.org/get-started/locally/
with pip install

pip install numpy
pip install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html

[mathandy]

I can't take credit for this, or remember where I found it, but what worked for me was

pip uninstall torch
pip install torch==1.7.0+cu110 torchvision==0.8.1+cu110 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

[hszhoushen]

I can't take credit for this, or remember where I found it, but what worked for me was

pip uninstall torch
pip install torch==1.7.0+cu110 torchvision==0.8.1+cu110 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

It works!

[justopit]

Hi! I have got several 3080 GPUs. However, I find some problems.
The system is Ubuntu 16.04, the version of PyTorch is 1.7.0+cu110
My codes are:
import torch ---(1)
X = torch.rand((3,6)) ---(2)
X = X.cuda() ---(3)

I find the step (1) and (2) very smooth, which means it nearly does not cost any time.
However, when I set CUDA_VISIBLE_DEVICES=1,
I found the third step costs much time to finish moving the data from memory to VRAM. But, when I set CUDA_VISIBLE_DEVICES=0, it is also smooth.

[WangWenhao0716]

Hi! I have got several 3080 GPUs. However, I find some problems.
The system is Ubuntu 16.04, the version of PyTorch is 1.7.0+cu110
My codes are:
import torch ---(1)
X = torch.rand((3,6)) ---(2)
X = X.cuda() ---(3)

I find the step (1) and (2) very smooth, which means it nearly does not cost any time.
However, when I set CUDA_VISIBLE_DEVICES=1,
I found the third step costs much time to finish moving the data from memory to VRAM. But, when I set CUDA_VISIBLE_DEVICES=0, it is also smooth.

It is a very old problem. And PyTorch support 3080 and 3090 officially since 1.7.0.

[rojas70]

Greetings,

The problem I will note below only happens when I debug with code but not on a python terminal. I believe my setup is properly loaded. Below all info necessary.

Error message:

NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.

Environment Information: my setup tries to follow the Nvidia compatibility matrix: driver-470/toolkit 114/libcudnn8.2/pytorch1.9+cu111

---

PyTorch version: 1.9.0+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.11.0-25-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 11.4.100
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 470.57.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.0
[pip3] torch==1.9.0+cu111 <---- Is this a problem? No cuda 114?
[pip3] torchaudio==0.9.0
[pip3] torchvision==0.10.0+cu111
[conda] Could not collect

---

If I run the code below in a python or python interpreter I get no errors:

import torch
X = torch.rand((3,6))
X = X.cuda()

It only happen inside Microsoft Code. I have double checked the correct interpreter/branch is loaded... But continue to get the same error.

I also tried many other combinations of drivers/toolkits/cudnn/pytorch but have not been able to get this right. It's killing me. Any pointers please?

[johndpope]

" when I debug with code but not on a python terminal."
I suspect vscode is picking up wrong environment.
you can select environment - see here
https://code.visualstudio.com/docs/python/environments

( I highly recommend miniconda - it integrates within vscode.
If you are on unix - use oh-my-zsh + zsh / https://github.com/ohmyzsh/ohmyzsh
it gives you a sanity check and clarity at terminal on which conda environment you're using. )

[Heermosi]

Greetings,

The problem I will note below only happens when I debug with code but not on a python terminal. I believe my setup is properly loaded. Below all info necessary.

Error message:

NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.

Environment Information: my setup tries to follow the Nvidia compatibility matrix: driver-470/toolkit 114/libcudnn8.2/pytorch1.9+cu111

PyTorch version: 1.9.0+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.11.0-25-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 11.4.100
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 470.57.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.0
[pip3] torch==1.9.0+cu111 <---- Is this a problem? No cuda 114?
[pip3] torchaudio==0.9.0
[pip3] torchvision==0.10.0+cu111
[conda] Could not collect

If I run the code below in a python or python interpreter I get no errors:

import torch
X = torch.rand((3,6))
X = X.cuda()

It only happen inside Microsoft Code. I have double checked the correct interpreter/branch is loaded... But continue to get the same error.

I also tried many other combinations of drivers/toolkits/cudnn/pytorch but have not been able to get this right. It's killing me. Any pointers please?

I partly agree with johndpope, You need to check out the environment first
Type the following in your terminal started python and vs-code started python each

1. Check out the python lib path

import sys
sys.path

This shall show you the lib path python in use. If you see a different setting, then you might be reading from a different python lib

2. Check out the pytorch's version and support

import torch
torch.__VERSION__
torch.cuda.get_arch_list()

If you see any differences in the 2 printings you might be using a different python lib

As part of my suggestion.
Do not install pytorch on host, use the pre-built pytorch dockers instead!

[rojas70]

Thank you @johndpope and @Heermosi

I have continued to study my environment and continue to think it is properly setup in code...
I wanted to share a few other results to see if it might possibly indicate some other culprit.

Currently, inside code I can run:

import torch
X = torch.rand((3,6))
X = X.cuda()

and/or

device = torch.device("cuda:0")
x = torch.zeros(1, dtype=torch.float32, requires_grad=True).to(device)

But if I try to do:

x= torch.zeros(1, dtype=torch.float32, requires_grad=True, device='cuda')

Then, I get the error

Traceback (most recent call last):
  File "<string>", line 1, in <module>
RuntimeError: CUDA error: no kernel image is available for execution on the device

Why might passing the device inside the torch call cause such an error but not outside?

[rojas70]

Thank you for the feedback.

There were indeed problems in my environment. Packages were not being saved locally but globally... I started the virtual environment from scratch again and verified each installation and software version to ensure consistency.

I came up with the following set of instructions in case it might help others:

Nvidia Driver and CUDA Toolkit

1. If already installed, examine your Nvidia GPU driver version

nvidia-smi
or
cat /proc/driver/nvidia/version

2. Learn its architecture
   sudo lshw -C display

3. Learn your current Linux kernel
   uname -a

4. Look up the Nvidia Compatibility Matrix to determine the correct driver, toolkit, and libcudnn
   https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html
   https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html#cudnn-versions-linux (gcc, glibc)

5. Install Driver
   sudo apt install nvidia-driver-XXX

6. Install CUDA Toolkit
   https://developer.nvidia.com/cuda-downloads

7. Install libcudnnX (useful to do deep learning with cuda)
   https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html
   sudo apt install libcudnnX

8. Install pytorch

• we will wait for this undtil you setup your virtualenv below.

Testing your system’s python setup

1. First note, the location of the system-wide python interpreter
   which python3

2. Note the location of teh system-wide pip
   which pip3

3. What packages are there globally (this command will also list packages that were installed via apt-get install)
   python3 -m pip list (or alternatively python3 -m pip freeze)

4. Create virtualenv if not yet created
   python3 -m venv name_for_your_env

5. Usually, you will be asked to install the required files; normally the file “requirements.txt”. Examine it and become familiar with it. From within your virtual environment, install them via:

python3 -m pip install -r requirements.txt

7. If not already installed, install pytorch.
   You can get the pip3/conda command from here. Most people recommend conda/docker installs. We are doing pip3 to have more flexiblity with the packages we need with different repos.

• go to https://pytorch.org/ (choose your config). You may get a command like:

8. Note that if a package is properly installed, it should appear in your virtual_env/lib/pythonX.X/site-packages forlder.

9. Additionally, ensure your pythonpath is properly set (learn more about pythonpath/imports/sys.path here: https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html)

• pythonpath is a environment variable that contains paths to load python modules/scripts that are not binaries (i.e. located.
• The pythonpath env variable is set in the .bashrc file found in your user folder (the user folder is located at ~/ and the “.” means it is a hidden file). Use your favorite editor to open it:

emacs ~/.bashrc

• Look to see if you already set any pythonpath’s.
  export PYTHONPATH=$PYTHONPATH:/new/path1/goes/here:/new/path2/goes/here:

Sanity Checks for torch/gpu

1. In your virtualenv, open a python interpreter:

python3 (or even better ipython3 -- you will need to install first pip3 install ipython).

2. Check the system path from which modules are loaded

import sys
sys.path (should not see undesired paths here).

2. Import torch

impor torch

3. Double check that this torch module is located inside your virtual environment

import imp
imp.find_module(‘torch’) --> should return a path in your virtualenv

4. Check the version of your torch module and cuda

torch.version
torch.version.cuda

5. Check the supported architectures
   torch.cuda.get_arch_list()

6. Check for the number of gpu detected

torch.cuda.device_count()

7. Can you read the device?

device=torch.device('cuda:0') # 0 by default, if you have more gpu’s increase your index.

[MaxGCM]

ys.platform: linux
Python: 3.6.9 (default, Oct 8 2020, 12:12:24) [GCC 8.4.0]
CUDA available: True
GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU
CUDA_HOME: /usr/local/cuda-11.1
NVCC: Build cuda_11.1.TC455_06.29069683_0
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.8.1+cu111
PyTorch compiling details: PyTorch built with:

• GCC 7.3
• C++ Version: 201402
• Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 11.1
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
• CuDNN 8.0.5
• Magma 2.5.2
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.1+cu111
OpenCV: 4.5.2
MMCV: 1.2.6
MMCV Compiler: GCC 7.5
MMCV CUDA Compiler: 11.1
MMDetection: 2.9.0
MMDetection3D: 0.10.0+

the nvidia-smi is

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.86 Driver Version: 470.86 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A |
| N/A 49C P8 14W / N/A | 577MiB / 5938MiB | 2% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+

when i trained, i got the RuntimeError: CUDA error: no kernel image is available for execution on the device.
when i trained and compiled, i tried TORCH_CUDA_ARCH_LIST=8.6, but it was still not worked. Someone who also met this issue??

[WangWenhao0716]

Please install it using conda or pip, do Not compile by yourself. 发自我的iPhone

…

------------------ Original ------------------ From: Max ***@***.***&gt; Date: Sat,Jan 15,2022 8:57 PM To: pytorch/pytorch ***@***.***&gt; Cc: Wenhao Wang ***@***.***&gt;, State change ***@***.***&gt; Subject: Re: [pytorch/pytorch] The support for 3080 or 3090 (#45021) sys.platform: linux Python: 3.6.9 (default, Oct 8 2020, 12:12:24) [GCC 8.4.0] CUDA available: True GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU CUDA_HOME: /usr/local/cuda-11.1 NVCC: Build cuda_11.1.TC455_06.29069683_0 GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.8.1+cu111 PyTorch compiling details: PyTorch built with: GCC 7.3 C++ Version: 201402 Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683) OpenMP 201511 (a.k.a. OpenMP 4.5) NNPACK is enabled CPU capability usage: AVX2 CUDA Runtime 11.1 NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86 CuDNN 8.0.5 Magma 2.5.2 Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, TorchVision: 0.9.1+cu111 OpenCV: 4.5.2 MMCV: 1.2.6 MMCV Compiler: GCC 7.5 MMCV CUDA Compiler: 11.1 MMDetection: 2.9.0 MMDetection3D: 0.10.0+ when i trained, i just got the meesage RuntimeError: CUDA error: no kernel image is available for execution on the device,i tried when i compiled with the TORCH_CUDA_ARCH_LIST=8.6, it is still not working, someone who met this issues?? — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you modified the open/close state.Message ID: ***@***.***&gt;

[MaxGCM]

Please install it using conda or pip, do Not compile by yourself. 发自我的iPhone

Thanks for replying, do you mean i install that the mmdetection3d using conda or pip? is that right?

[WangWenhao0716]

for mmdetection3d please according to its own instruction. 发自我的iPhone

…

------------------ Original ------------------ From: Max ***@***.***&gt; Date: Sat,Jan 15,2022 10:10 PM To: pytorch/pytorch ***@***.***&gt; Cc: Wenhao Wang ***@***.***&gt;, State change ***@***.***&gt; Subject: Re: [pytorch/pytorch] The support for 3080 or 3090 (#45021) Please install it using conda or pip, do Not compile by yourself. 发自我的iPhone … ------------------ Original ------------------ From: Max @.&gt; Date: Sat,Jan 15,2022 8:57 PM To: pytorch/pytorch @.&gt; Cc: Wenhao Wang @.&gt;, State change @.&gt; Subject: Re: [pytorch/pytorch] The support for 3080 or 3090 (#45021) sys.platform: linux Python: 3.6.9 (default, Oct 8 2020, 12:12:24) [GCC 8.4.0] CUDA available: True GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU CUDA_HOME: /usr/local/cuda-11.1 NVCC: Build cuda_11.1.TC455_06.29069683_0 GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.8.1+cu111 PyTorch compiling details: PyTorch built with: GCC 7.3 C++ Version: 201402 Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683) OpenMP 201511 (a.k.a. OpenMP 4.5) NNPACK is enabled CPU capability usage: AVX2 CUDA Runtime 11.1 NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86 CuDNN 8.0.5 Magma 2.5.2 Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, TorchVision: 0.9.1+cu111 OpenCV: 4.5.2 MMCV: 1.2.6 MMCV Compiler: GCC 7.5 MMCV CUDA Compiler: 11.1 MMDetection: 2.9.0 MMDetection3D: 0.10.0+ when i trained, i just got the meesage RuntimeError: CUDA error: no kernel image is available for execution on the device,i tried when i compiled with the TORCH_CUDA_ARCH_LIST=8.6, it is still not working, someone who met this issues?? — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you modified the open/close state.Message ID: @.***&gt; Thanks for replying, do you mean i install that the mmdetection3d using conda or pip? is that right? — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you modified the open/close state.Message ID: ***@***.***&gt;