RuntimeError: /bevfusion-main/mmdet3d/ops/spconv/include/tensorview/helper_launch.h 17 N > 0 assert faild. CUDA kernel launch blocks must be positive, but got N= 0

[youngfly]

hello， i met a very strange problem as tiltle during the training.
This problem occurs in different epoch.
1)

problem

my CUDA version is 11.1.
GPU:Nvidia A100
pytorch:1.9.0
other required libraries are installed indicated in your README

[kentang-mit]

According to some previous discussions, it is possible that the error is caused by unmatched CUDA versions (between the OS and the anaconda environment). Would you mind checking what is the cuda-toolkit version you installed in anaconda?
By the way, we are going to replace the spconv backend in the next release. Please stay tuned.

[youngfly]

I have sovled this problem with a new environment. It is very strange.