ninja: build stopped: subcommand failed.

[sxdai]

🐛 Bug

[446/3104] Building CXX object third_party/fbgemm/CMakeFiles/fbgemm_avx2.dir/src/FbgemmI8DepthwiseAvx2.cc.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "setup.py", line 719, in
build_deps()
File "setup.py", line 285, in build_deps
build_dir='build')
File "/home/prjs/tmp/pytorch/tools/build_pytorch_libs.py", line 278, in build_caffe2
check_call(ninja_cmd, cwd=build_dir, env=my_env)
File "/root/miniconda3/envs/py35/lib/python3.5/subprocess.py", line 271, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ninja', 'install']' returned non-zero exit status 1

[umanwizard]

Please provide the full logs, this does not contain the actual compile error.

[yiling-chen]

I ran into the same problem and the following is my full log.

Building wheel torch-1.1.0a0+df05c7f
-- Building version 1.1.0a0+df05c7f
[3/784] Building CXX object caffe2/CMakeFiles/caffe2.dir/ideep/operators/conv_op.cc.o
FAILED: caffe2/CMakeFiles/caffe2.dir/ideep/operators/conv_op.cc.o
/usr/bin/c++ -DCPUINFO_SUPPORTED_PLATFORM=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DNNP_CONVOLUTION_ONLY=0 -DNNP_INFERENCE_ONLY=0 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DUSE_C11_ATOMICS=1 -D_FILE_OFFSET_BITS=64 -Dcaffe2_EXPORTS -I../aten/src -I. -I../ -I../third_party/protobuf/src -I../cmake/../third_party/benchmark/include -Icaffe2/contrib/aten -I../third_party/onnx -Ithird_party/onnx -I../third_party/foxi -Ithird_party/foxi -Icaffe2/aten/src/TH -I../aten/src/TH -Icaffe2/aten/src -Iaten/src -I../aten/../third_party/catch/single_include -I../aten/src/ATen/.. -Icaffe2/aten/src/ATen -I../caffe2/core/nomnigraph/include -I../third_party/miniz-2.0.8 -I../c10/.. -I../third_party/QNNPACK/include -I../third_party/pthreadpool/include -I../third_party/NNPACK/include -I../third_party/cpuinfo/include -I../third_party/fbgemm/include -I../third_party/fbgemm -I../third_party/FP16/include -isystem /home/yilingchen/miniconda3/envs/mrcnn-onnx/include -isystem ../third_party/ideep/include -isystem ../torch/include -isystem third_party/gloo -isystem ../cmake/../third_party/gloo -isystem ../cmake/../third_party/googletest/googlemock/include -isystem ../cmake/../third_party/googletest/googletest/include -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -isystem ../third_party -isystem ../cmake/../third_party/eigen -isystem /home/yilingchen/miniconda3/envs/mrcnn-onnx/include/python3.7m -isystem /home/yilingchen/miniconda3/envs/mrcnn-onnx/lib/python3.7/site-packages/numpy/core/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include -isystem include -fvisibility-inlines-hidden -fopenmp -D_FORCE_INLINES -D_MWAITXINTRIN_H_INCLUDED -D__STRICT_ANSI__ -DUSE_FBGEMM -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -fPIC -DCAFFE2_USE_GLOO -DHAVE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -fvisibility=hidden -DCAFFE2_BUILD_MAIN_LIB -O2 -pthread -std=gnu++11 -MD -MT caffe2/CMakeFiles/caffe2.dir/ideep/operators/conv_op.cc.o -MF caffe2/CMakeFiles/caffe2.dir/ideep/operators/conv_op.cc.o.d -o caffe2/CMakeFiles/caffe2.dir/ideep/operators/conv_op.cc.o -c ../caffe2/ideep/operators/conv_op.cc
../caffe2/ideep/operators/conv_op.cc: In member function ‘virtual bool caffe2::IDEEPConvOp::RunOnDeviceWithOrderNCHW()’:
../caffe2/ideep/operators/conv_op.cc:75:27: error: no matching function for call to ‘ideep::convolution_forward::expected_weights_descriptor(ideep::param::dims, mkldnn::memory::data_type, std::vector&, ideep::param::dims, ideep::param::dims, std::vector&, int&, caffe2::IDEEPConvOp::ialgo&, caffe2::IDEEPConvOp::iprop&, mkldnn::memory::data_type, ideep::param::dims)’
X.get_dims());
^
In file included from ../third_party/ideep/include/ideep.hpp:44:0,
from ../caffe2/ideep/ideep_utils.h:6,
from ../caffe2/ideep/operators/conv_pool_base_op.h:6,
from ../caffe2/ideep/operators/conv_op.cc:1:
../third_party/ideep/include/ideep/computations.hpp:2173:29: note: candidate: static ideep::param::descriptor ideep::convolution_forward::expected_weights_descriptor(const dims&, ideep::param::data_type, const dims&, const dims&, const dims&, const dims&, int, ideep::algorithm, ideep::prop_kind, ideep::param::data_type)
static tensor::descriptor expected_weights_descriptor(
^~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/ideep/include/ideep/computations.hpp:2173:29: note: candidate expects 10 arguments, 11 provided
../third_party/ideep/include/ideep/computations.hpp:593:22: note: candidate: ideep::param::descriptor ideep::primitive_group::expected_weights_descriptor() const
tensor::descriptor expected_weights_descriptor() const {
^~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/ideep/include/ideep/computations.hpp:593:22: note: candidate expects 0 arguments, 11 provided
[36/784] Building CXX object caffe2/CMakeFiles/caffe2.dir/operators/cosh_op.cc.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "setup.py", line 719, in
build_deps()
File "setup.py", line 285, in build_deps
build_dir='build')
File "/mnt/Data-1/Projects/faster-rcnn-onnx/pytorch/tools/build_pytorch_libs.py", line 278, in build_caffe2
check_call(ninja_cmd, cwd=build_dir, env=my_env)
File "/home/yilingchen/miniconda3/envs/onnxrt/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ninja', 'install']' returned non-zero exit status 1.

[soumith]

@yiling-chen please open a new issue using this template: https://github.com/pytorch/pytorch/issues/new?template=bug-report.md

[deisler134]

@soumith when I build libtorch meet the same issue:

caffe2/CMakeFiles/caffe2.dir//aten/src/ATen/native/cpu/Activation.cpp.DEFAULT.cpp.DEFAULT.cpp.DEFAULT.cpp.o:(.data.rel.local+0x0): first defined here
caffe2/CMakeFiles/caffe2.dir//aten/src/ATen/native/cpu/Activation.cpp.DEFAULT.cpp.o:(.data.rel.local+0x0): multiple definition of `at::native::DispatchStub<void (*)(at::TensorIterator&, c10::Scalar, c10::Scalar), at::native::threshold_stub>::DEFAULT'
caffe2/CMakeFiles/caffe2.dir/__/aten/src/ATen/native/cpu/Activation.cpp.DEFAULT.cpp.DEFAULT.cpp.DEFAULT.cpp.o:(.data.rel.local+0x0): first defined here
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "../tools/build_libtorch.py", line 22, in
build_python=False, rerun_cmake=True, build_dir='.')
File "/media/deisler/Data/project/pytorch/tools/build_pytorch_libs.py", line 293, in build_caffe2
check_call(ninja_cmd, cwd=build_dir, env=my_env)
File "/home/deisler/anaconda3/envs/py36/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ninja', 'install']' returned non-zero exit status 1.

[chopwoodwater]

I met the same issue as well

[chopwoodwater]

@deisler134 @yiling-chen Have you solved it? I have exactly the same error log as yours.

@soumith is there any steps we can take to solve it?

[mrDarkk]

I have same problem.

ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "setup.py", line 756, in
build_deps()
File "setup.py", line 321, in build_deps
cmake=cmake)
File "/Users/bhupendrabanothe/pytorch/tools/build_pytorch_libs.py", line 63, in build_caffe2
cmake.build(my_env)
File "/Users/bhupendrabanothe/pytorch/tools/setup_helpers/cmake.py", line 330, in build
self.run(build_args, my_env)
File "/Users/bhupendrabanothe/pytorch/tools/setup_helpers/cmake.py", line 143, in run
check_call(command, cwd=self.build_dir, env=env)
File "/anaconda3/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 1.

[XuShoweR]

I have same problem.

ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "setup.py", line 756, in
build_deps()
File "setup.py", line 321, in build_deps
cmake=cmake)
File "/Users/bhupendrabanothe/pytorch/tools/build_pytorch_libs.py", line 63, in build_caffe2
cmake.build(my_env)
File "/Users/bhupendrabanothe/pytorch/tools/setup_helpers/cmake.py", line 330, in build
self.run(build_args, my_env)
File "/Users/bhupendrabanothe/pytorch/tools/setup_helpers/cmake.py", line 143, in run
check_call(command, cwd=self.build_dir, env=env)
File "/anaconda3/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 1.

Did you solve the problem.I have same problem.

[sonuagarwal1008]

**

I am getting below error while installing Caffe2 on windows 10 without CUDA support.

**
See also "C:/Users/agarwals/Downloads/PortableGit/pytorch/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/agarwals/Downloads/PortableGit/pytorch/build/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
File "tools\build_libtorch.py", line 23, in
rerun_cmake=True, cmake_only=False, cmake=CMake())
File "C:\Users\agarwals\Downloads\PortableGit\pytorch\tools\build_pytorch_libs.py", line 59, in build_caffe2
rerun_cmake)
File "C:\Users\agarwals\Downloads\PortableGit\pytorch\tools\setup_helpers\cmake.py", line 321, in generate
self.run(args, env=my_env)
File "C:\Users\agarwals\Downloads\PortableGit\pytorch\tools\setup_helpers\cmake.py", line 141, in run
check_call(command, cwd=self.build_dir, env=env)
File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-GNinja', '-DBUILDING_WITH_TORCH_LIBS=OFF', '-DBUILD_BINARY=ON', '-DBUILD_PYTHON=False', '-DBUILD_SHARED_LIBS=OFF', '-DBUILD_TEST=True', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_GENERATOR=Ninja', '-DCMAKE_INSTALL_PREFIX=C:\Users\agarwals\Downloads\PortableGit\pytorch\torch', '-DCMAKE_PREFIX_PATH=C:\ProgramData\Anaconda3\Lib\site-packages', '-DCMAKE_VERBOSE_MAKEFILE=1', '-DMSVC_Z7_OVERRIDE=OFF', '-DNUMPY_INCLUDE_DIR=C:\ProgramData\Anaconda3\lib\site-packages\numpy\core\include', '-DONNX_NAMESPACE=onnx_c2', '-DPYTHON_EXECUTABLE=C:\ProgramData\Anaconda3\python.exe', '-DPYTHON_INCLUDE_DIR=C:\ProgramData\Anaconda3\include', '-DUSE_CUDA=OFF', '-DUSE_NUMPY=True', '-DUSE_OBSERVERS=OFF', 'C:\Users\agarwals\Downloads\PortableGit\pytorch']' returned non-zero exit status 1.
"Caffe2 building failed"

[destinyzs]

The same problem

[surak]

Any solution for this?

[3shv]

I noticed similar error on WSL2 when the compilation hit memory limits. I was able to solve by increasing the RAM

[laowalter]

me too, now, when python setup.py install