[Bug] SurroundOcc bug: error in ms_deformable_col2im_cuda: invalid configuration argument

[PKUHaoWang]

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• I have read the FAQ documentation but cannot get the expected help.
• The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I have modified the scripts/configs, or I'm working on my own tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

sys.platform: linux
Python: 3.8.19 (default, Mar 20 2024, 19:58:24) [GCC 11.2.0]
CUDA available: True
numpy_random_seed: 2147483648
GPU 0,1: NVIDIA GeForce RTX 4090
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.8, V11.8.89
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
PyTorch: 1.11.0+cu113
PyTorch compiling details: PyTorch built with:

• GCC 7.3
• C++ Version: 201402
• Intel(R) oneAPI Math Kernel Library Version 2023.1-Product Build 20230303 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v2.5.2 (Git Hash a9302535553c73243c632ad3c4c80beec3d19a1e)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 11.3
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
• CuDNN 8.2
• Magma 2.5.2
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.12.0+cu113
OpenCV: 4.9.0
MMEngine: 0.8.0

Reproduces the problem - code sample

python -m torch.distributed.launch --nproc_per_node=1 tools/train.py configs/occupancy/mv-occ_surroundocc.py --work-dir=work_dirs/mv_occ --launcher="pytorch"

Reproduces the problem - command or script

python -m torch.distributed.launch --nproc_per_node=1 tools/train.py configs/occupancy/mv-occ_surroundocc.py --work-dir=work_dirs/mv_occ --launcher="pytorch"

Reproduces the problem - error message

error in ms_deformable_col2im_cuda: invalid configuration argument
error in ms_deformable_im2col_cuda: invalid configuration argument

Additional information

I find that before deformable attention in SurroundOcc, the num_query = 0.

Could you release your SurroundOcc config or code?

Thanks a lot !

[Tai-Wang]

We will try to release the code for SurroundOcc this month.

[mxh1999]

@WH0226 Hello, I implemented SurroundOcc based on EmbodiedScan here. I hope this can help.

[Tai-Wang]

Close this issue for now due to inactivity. Please feel free to reopen it and post new questions if necessary.

[PKUHaoWang]

Thanks a lot!

[PKUHaoWang]

@mxh1999 Hello, we conducted experiments based on this code and found that the experimental results cannot achieve the effect in the paper. Could you provide more detailed parameter settings or checkpoints?

[mxh1999]

@mxh1999 Hello, we conducted experiments based on this code and found that the experimental results cannot achieve the effect in the paper. Could you provide more detailed parameter settings or checkpoints?

Since it's been a long time, finding it may be difficult.
Could I ask what is wrong with the results?

[PKUHaoWang]

@mxh1999 Hello, we conducted experiments based on this code and found that the experimental results cannot achieve the effect in the paper. Could you provide more detailed parameter settings or checkpoints?

Since it's been a long time, finding it may be difficult. Could I ask what is wrong with the results?

We found that the mIOU value is around 0.006-0.007, which is difficult to improve further. If it is convenient, can we use email or WeChat to communicate? Mine is haowang226@163.com.

[mxh1999]

@WH0226 I have uploaded the configs, logs and checkpoints of Surroundocc to GoogleDrive. I hope this can help.

[PKUHaoWang]

@WH0226 I have uploaded the configs, logs and checkpoints of Surroundocc to GoogleDrive. I hope this can help.

Thanks a lot!