instructions on how to successfully build spconv 1.0

[MartinHahner]

Make sure you are running the following commands on a host with a Pascal GPU.
(Newer architectures probably also works.)

conda create --name spconv-1.0 python=3.6 pytorch=1.1 cudatoolkit=9.2 cudatoolkit-dev=9.2 \
cmake --channel pytorch --channel=conda-forge
conda activate spconv-1.0
conda install cudnn
conda install boost
git clone https://github.com/traveller59/spconv spconv_8da6f96 --recursive
cd spconv_8da6f96
git checkout 8da6f967fb9a054d8870c3515b1b44eca2103634

If necessary:
Download and extract
0001-Allow-to-specifiy-CUDA_ROOT-directory-and-pick-corre.patch.zip
and patch spconv via:
git am <PATH_TO_EXTRACTED_FILE>/0001-Allow-to-specifiy-CUDA_ROOT-directory-and-pick-corre.patch

(or fix issue #78 by yourself)

CUDA_ROOT=<PATH_TO_YOUR_CONDA_INSTALLATION>/conda_envs/spconv-1.0 python setup.py bdist_wheel
cd dist/
pip install *

Test spconv via:
python -c 'import spconv'
(should just return and not raise any errors)

[dekura]

Make sure you are running the following commands on a host with a Pascal GPU.
(Newer architectures probably also works.)

conda create --name spconv-1.0 python=3.6 pytorch=1.1 cudatoolkit=9.2 cudatoolkit-dev=9.2 \
cmake --channel pytorch --channel=conda-forge
conda activate spconv-1.0
conda install cudnn
conda install boost
git clone https://github.com/traveller59/spconv spconv_8da6f96 --recursive
cd spconv_8da6f96
git checkout 8da6f967fb9a054d8870c3515b1b44eca2103634

If necessary:
Download and extract
0001-Allow-to-specifiy-CUDA_ROOT-directory-and-pick-corre.patch.zip
and patch spconv via:
git am <PATH_TO_EXTRACTED_FILE>/0001-Allow-to-specifiy-CUDA_ROOT-directory-and-pick-corre.patch

CUDA_ROOT=<PATH_TO_YOUR_CONDA_INSTALLATION>/conda_envs/spconv-1.0 python setup.py bdist_wheel
cd dist/
pip install *

Test spconv via:
python -c 'import spconv'
(should just return and not raise any errors)

@MartinHahner88 Thank you for the detailed instruction~
I successfully build spconv 1.0.
Note that the gcc must > 4.9

[ConanCui]

This method is easy to work, good job.

[sarimmehdi]

These instructions do not work. I still have the same problem as described here: open-mmlab/OpenPCDet#187 (comment)

[sarimmehdi]

Hello. Is there any way to do this in Google Colab? I tried but am unable to. Please let me know, thanks!

[traveller59]

@sarimmehdi
run following command in colab:

!pip install cmake --upgrade && PROBLEM_FILE=/usr/local/lib/python3.6/dist-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake && \
    sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces;-fopenmp//g' $PROBLEM_FILE && \
    sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces//g' $PROBLEM_FILE && \
  rm -rf spconv && git clone https://github.com/traveller59/spconv --recursive && cd ./spconv && python setup.py install

[Ultra11nstinct]

Hi @traveller59
you did a really great work.

and
@MartinHahner
you contribute a lot on how to build PCDet and spconv.

i have built spconv following your instructions successfully.
BUT there is a wierd thing that i clone the spconv1.0 repo as you mentioned.
git clone https://github.com/traveller59/spconv spconv_8da6f96 --recursive
and when it finish, it came with this
Processing ./spconv-1.2.1-cp36-cp36m-linux_x86_64.whl
Installing collected packages: spconv
Successfully installed spconv-1.2.1

i use sudo apt-get install libboost-all-dev for boost installation.
and didnt modify the setup.py
I wonder why i finally built spconv 1.2 instead of spconv 1.0
you have any idea about this issue?

P.S.:
i have another question
i saw you create conda env. with specifying cuda version==9.0.
is that a neccesary thing?
because i found that even if i specify the cuda version when creating conda env., the cuda_path are still link to usr/local/cuda (cuda_path to cuda10.2 installed by manual)
sorry i'm new in this area.
maybe these questions are simple.

Here is my env. info
python 3.6
pytorch 1.3
nvidia driver 440.33.01
cuda 10.2
cudnn 7.6

Sincerely looking forward for your reply.:)

[MartinHahner]

I wonder why i finally built spconv 1.2 instead of spconv 1.0
you have any idea about this issue?

After
git clone https://github.com/traveller59/spconv spconv_8da6f96 --recursive
you also need to run

cd spconv_8da6f96
git checkout 8da6f967fb9a054d8870c3515b1b44eca2103634

Otherwise you will work with the latest commit on the master branch instead with commit 8da6f96 (spconv 1.0).

i saw you create conda env. with specifying cuda version==9.0.
is that a neccesary thing?

I did not specify CUDA 9.0, but CUDA 9.2, I don't know if you can successfully build spconv 1.0 with CUDA 10.2.
Hope that helps.

[Ultra11nstinct]

Thank you for your reply @MartinHahner
feel sorry for the mistakes in my reply.
i just,emmm confusion in trying install different versions of CUDA, CUDNN anaconda......
sorry about that.

As you adviced,
i have turn spconv to branch(or commit?) 8da6f96 by using

cd spconv_8da6f96
git checkout 8da6f967fb9a054d8870c3515b1b44eca2103634

finally, i successfully build spconv 1.0 with CUDA 10.2.
Thank you again for your reply.

i still wonder why when creating conda env, it automatically install cuda toolkit and cudnn.
but it seems that the cuda and cudnn installed by conda won't work when compiling or build some packs.

[MartinHahner]

It says

unsupported GNU version! gcc versions later than 7 are not supported!

so, try to run srun gcc --version to verify that you are using a gcc version not later than 7

[RolandoAvides]

I have currently the V9.2

[RolandoAvides]

Problem solved.

[MartinHahner]

how?

[RolandoAvides]

I was missing cmake in my computer. The instructions were great! Thanks!

[3bobo]

@MartinHahner , thanks for your instructions! I install spconv1.0 in my computer successfully. However, I want to install spconv1.0 in our lab server, which I do not have sudo permissions, so I just install libboost in my conda environment, so how to specify the libboost path to install spconv1.0?

[MartinHahner]

maybe thread #78 can help

[3bobo]

maybe thread #78 can help

Problem solved. I changed the spconv1.0/third_party/pybind11/tests/CMakeLists.txt to find the libboost path.
Before:
find_package(Boost 1.56)
After:
find_package(Boost 1.56 HINTS /anaconda_path_to_libboost/include/)

[wj-shin]

thanks for your effort

I do this way today.

but I can not install v1.0 so can you give me some tip?

Sorry

[shadismz]

@sarimmehdi run following command in colab:

!pip install cmake --upgrade && PROBLEM_FILE=/usr/local/lib/python3.6/dist-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake && \
    sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces;-fopenmp//g' $PROBLEM_FILE && \
    sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces//g' $PROBLEM_FILE && \
  rm -rf spconv && git clone https://github.com/traveller59/spconv --recursive && cd ./spconv && python setup.py install

Hello,
I want to install spconv 1.2 in colab, I have the following problem for a while. I ran
!pip install cmake --upgrade && PROBLEM_FILE=/usr/local/lib/python3.7/dist-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces;-fopenmp//g' $PROBLEM_FILE && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces//g' $PROBLEM_FILE && \ rm -rf spconv && git clone https://github.com/traveller59/spconv --recursive && cd ./spconv && python setup.py install and also !pip install cmake --upgrade && PROBLEM_FILE=/usr/local/lib/python3.7/dist-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces;-fopenmp//g' $PROBLEM_FILE && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces//g' $PROBLEM_FILE && \ rm -rf spconv && git clone https://github.com/traveller59/spconv --recursive && cd ./spconv && python setup.py bdist_wheel but it was not successfull.
I get the foloowing error:
/content/spconv
running install
running bdist_egg
running egg_info
writing spconv.egg-info/PKG-INFO
writing dependency_links to spconv.egg-info/dependency_links.txt
writing top-level names to spconv.egg-info/top_level.txt
adding license file 'LICENSE'
writing manifest file 'spconv.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
Release
|||||CMAKE ARGS||||| ['-DCMAKE_PREFIX_PATH=/usr/local/lib/python3.7/dist-packages/torch', '-DPYBIND11_PYTHON_VERSION=3.7', '-DSPCONV_BuildTests=OFF', '-DPYTORCH_VERSION=11201', '-DCMAKE_CUDA_FLAGS="--expt-relaxed-constexpr" -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/content/spconv/build/lib.linux-x86_64-3.7/spconv', '-DCMAKE_BUILD_TYPE=Release']
-- Caffe2: CUDA detected: 11.1
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 11.1
-- Found cuDNN: v7.6.5 (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- /usr/local/cuda/lib64/libnvrtc.so shorthash is 3a20f2b6
-- Autodetected CUDA architecture(s): 7.5
-- Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75
CMake Warning at /usr/local/lib/python3.7/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
/usr/local/lib/python3.7/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
CMakeLists.txt:22 (find_package)

-- Autodetected CUDA architecture(s): 7.5
-- pybind11 v2.5.0
-- Configuring done
-- Generating done
-- Build files have been written to: /content/spconv/build/temp.linux-x86_64-3.7
Consolidate compiler generated dependencies of target cuhash
Consolidate compiler generated dependencies of target spconv_nms
[ 25%] Built target cuhash
[ 33%] Built target spconv_nms
Consolidate compiler generated dependencies of target spconv_utils
[ 41%] Built target spconv_utils
Consolidate compiler generated dependencies of target spconv
[ 45%] Building CUDA object src/spconv/CMakeFiles/spconv.dir/reordering.cu.o
In file included from /content/spconv/src/spconv/reordering.cu:18:0:
/content/spconv/include/spconv/reordering.cu.h:18:10: fatal error: THC/THCNumerics.cuh: No such file or directory
#include <THC/THCNumerics.cuh>
^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
src/spconv/CMakeFiles/spconv.dir/build.make:187: recipe for target 'src/spconv/CMakeFiles/spconv.dir/reordering.cu.o' failed
make[2]: *** [src/spconv/CMakeFiles/spconv.dir/reordering.cu.o] Error 1
CMakeFiles/Makefile2:179: recipe for target 'src/spconv/CMakeFiles/spconv.dir/all' failed
make[1]: *** [src/spconv/CMakeFiles/spconv.dir/all] Error 2
Makefile:135: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
File "setup.py", line 108, in
zip_safe=False,
File "/usr/local/lib/python3.7/dist-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 67, in run
self.do_egg_install()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/usr/lib/python3.7/distutils/command/install_lib.py", line 109, in build
self.run_command('build_ext')
File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "setup.py", line 48, in run
self.build_extension(ext)
File "setup.py", line 92, in build_extension
subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.

[3846chs]

@sarimmehdi run following command in colab:

!pip install cmake --upgrade && PROBLEM_FILE=/usr/local/lib/python3.6/dist-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake && \
    sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces;-fopenmp//g' $PROBLEM_FILE && \
    sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces//g' $PROBLEM_FILE && \
  rm -rf spconv && git clone https://github.com/traveller59/spconv --recursive && cd ./spconv && python setup.py install

Hello, I want to install spconv 1.2 in colab, I have the following problem for a while. I ran !pip install cmake --upgrade && PROBLEM_FILE=/usr/local/lib/python3.7/dist-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces;-fopenmp//g' $PROBLEM_FILE && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces//g' $PROBLEM_FILE && \ rm -rf spconv && git clone https://github.com/traveller59/spconv --recursive && cd ./spconv && python setup.py install and also !pip install cmake --upgrade && PROBLEM_FILE=/usr/local/lib/python3.7/dist-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces;-fopenmp//g' $PROBLEM_FILE && \ sed -i 's/-Wall;-Wextra;-Wno-unused-parameter;-Wno-missing-field-initializers;-Wno-write-strings;-Wno-unknown-pragmas;-Wno-missing-braces//g' $PROBLEM_FILE && \ rm -rf spconv && git clone https://github.com/traveller59/spconv --recursive && cd ./spconv && python setup.py bdist_wheel but it was not successfull. I get the foloowing error: /content/spconv running install running bdist_egg running egg_info writing spconv.egg-info/PKG-INFO writing dependency_links to spconv.egg-info/dependency_links.txt writing top-level names to spconv.egg-info/top_level.txt adding license file 'LICENSE' writing manifest file 'spconv.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_ext Release |||||CMAKE ARGS||||| ['-DCMAKE_PREFIX_PATH=/usr/local/lib/python3.7/dist-packages/torch', '-DPYBIND11_PYTHON_VERSION=3.7', '-DSPCONV_BuildTests=OFF', '-DPYTORCH_VERSION=11201', '-DCMAKE_CUDA_FLAGS="--expt-relaxed-constexpr" -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/content/spconv/build/lib.linux-x86_64-3.7/spconv', '-DCMAKE_BUILD_TYPE=Release'] -- Caffe2: CUDA detected: 11.1 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 11.1 -- Found cuDNN: v7.6.5 (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so) -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 3a20f2b6 -- Autodetected CUDA architecture(s): 7.5 -- Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75 CMake Warning at /usr/local/lib/python3.7/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message): static library kineto_LIBRARY-NOTFOUND not found. Call Stack (most recent call first): /usr/local/lib/python3.7/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found) CMakeLists.txt:22 (find_package)

-- Autodetected CUDA architecture(s): 7.5 -- pybind11 v2.5.0 -- Configuring done -- Generating done -- Build files have been written to: /content/spconv/build/temp.linux-x86_64-3.7 Consolidate compiler generated dependencies of target cuhash Consolidate compiler generated dependencies of target spconv_nms [ 25%] Built target cuhash [ 33%] Built target spconv_nms Consolidate compiler generated dependencies of target spconv_utils [ 41%] Built target spconv_utils Consolidate compiler generated dependencies of target spconv [ 45%] Building CUDA object src/spconv/CMakeFiles/spconv.dir/reordering.cu.o In file included from /content/spconv/src/spconv/reordering.cu:18:0: /content/spconv/include/spconv/reordering.cu.h:18:10: fatal error: THC/THCNumerics.cuh: No such file or directory #include <THC/THCNumerics.cuh> ^~~~~~~~~~~~~~~~~~~~~ compilation terminated. src/spconv/CMakeFiles/spconv.dir/build.make:187: recipe for target 'src/spconv/CMakeFiles/spconv.dir/reordering.cu.o' failed make[2]: *** [src/spconv/CMakeFiles/spconv.dir/reordering.cu.o] Error 1 CMakeFiles/Makefile2:179: recipe for target 'src/spconv/CMakeFiles/spconv.dir/all' failed make[1]: *** [src/spconv/CMakeFiles/spconv.dir/all] Error 2 Makefile:135: recipe for target 'all' failed make: *** [all] Error 2 Traceback (most recent call last): File "setup.py", line 108, in zip_safe=False, File "/usr/local/lib/python3.7/dist-packages/setuptools/init.py", line 153, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.7/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 67, in run self.do_egg_install() File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 109, in do_egg_install self.run_command('bdist_egg') File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 164, in run cmd = self.call_command('install_lib', warn_dir=0) File "/usr/local/lib/python3.7/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command self.run_command(cmdname) File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install_lib.py", line 11, in run self.build() File "/usr/lib/python3.7/distutils/command/install_lib.py", line 109, in build self.run_command('build_ext') File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "setup.py", line 48, in run self.build_extension(ext) File "setup.py", line 92, in build_extension subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp) File "/usr/lib/python3.7/subprocess.py", line 363, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.

I just commented out "#include <THC/THCNumerics.cuh>" where you got error (spconv/include/spconv/reordering.cu.h)

#include <THC/THCNumerics.cuh>

->

// #include <THC/THCNumerics.cuh>