Skip to content

Building Error #317

Description

@YOKE

Thanks for your great work! When I run

sudo pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" .
/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py:244: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.
cmdoptions.check_install_build_global(options)
Created temporary directory: /tmp/pip-ephem-wheel-cache-qbetfqhb
Created temporary directory: /tmp/pip-req-tracker-b7tfd9v5
Created requirements tracker '/tmp/pip-req-tracker-b7tfd9v5'
Created temporary directory: /tmp/pip-install-oj0_ps30
Processing /media/lab324/sdc/ybb/apex
Created temporary directory: /tmp/pip-req-build-i31tt77s
Added file:///media/lab324/sdc/ybb/apex to build tracker '/tmp/pip-req-tracker-b7tfd9v5'
Running setup.py (path:/tmp/pip-req-build-i31tt77s/setup.py) egg_info for package from file:///media/lab324/sdc/ybb/apex
Running command python setup.py egg_info
torch.version = 1.0.1.post2
running egg_info
creating pip-egg-info/apex.egg-info
writing pip-egg-info/apex.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/apex.egg-info/dependency_links.txt
writing top-level names to pip-egg-info/apex.egg-info/top_level.txt
writing manifest file 'pip-egg-info/apex.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found

reading manifest file 'pip-egg-info/apex.egg-info/SOURCES.txt'
writing manifest file 'pip-egg-info/apex.egg-info/SOURCES.txt'

Source in /tmp/pip-req-build-i31tt77s has version 0.1, which satisfies requirement apex==0.1 from file:///media/lab324/sdc/ybb/apex
Removed apex==0.1 from file:///media/lab324/sdc/ybb/apex from build tracker '/tmp/pip-req-tracker-b7tfd9v5'
Skipping bdist_wheel for apex, due to binaries being disabled for it.
Installing collected packages: apex
Created temporary directory: /tmp/pip-record-pc5mw3el
Running command /usr/bin/python3 -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-req-build-i31tt77s/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' --cpp_ext --cuda_ext install --record /tmp/pip-record-pc5mw3el/install-record.txt --single-version-externally-managed --compile
torch.version = 1.0.1.post2

Compiling cuda extensions with
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61
from /usr/local/cuda/bin


Warning:  Cuda extensions are being compiled with a version of Cuda that does not match the version used to compile Pytorch binaries.

Pytorch binaries were compiled with Cuda 9.0.176

running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/apex
copying apex/__init__.py -> build/lib.linux-x86_64-3.6/apex
creating build/lib.linux-x86_64-3.6/apex/normalization
copying apex/normalization/fused_layer_norm.py -> build/lib.linux-x86_64-3.6/apex/normalization
copying apex/normalization/__init__.py -> build/lib.linux-x86_64-3.6/apex/normalization
creating build/lib.linux-x86_64-3.6/apex/multi_tensor_apply
copying apex/multi_tensor_apply/multi_tensor_apply.py -> build/lib.linux-x86_64-3.6/apex/multi_tensor_apply
copying apex/multi_tensor_apply/__init__.py -> build/lib.linux-x86_64-3.6/apex/multi_tensor_apply
creating build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/handle.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/_amp_state.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/frontend.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/utils.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/opt.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/compat.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/_process_optimizer.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/rnn_compat.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/amp.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/__version__.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/__init__.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/wrap.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/scaler.py -> build/lib.linux-x86_64-3.6/apex/amp
copying apex/amp/_initialize.py -> build/lib.linux-x86_64-3.6/apex/amp
creating build/lib.linux-x86_64-3.6/apex/RNN
copying apex/RNN/cells.py -> build/lib.linux-x86_64-3.6/apex/RNN
copying apex/RNN/models.py -> build/lib.linux-x86_64-3.6/apex/RNN
copying apex/RNN/__init__.py -> build/lib.linux-x86_64-3.6/apex/RNN
copying apex/RNN/RNNBackend.py -> build/lib.linux-x86_64-3.6/apex/RNN
creating build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/sync_batchnorm_kernel.py -> build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/optimized_sync_batchnorm.py -> build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/__init__.py -> build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/sync_batchnorm.py -> build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/multiproc.py -> build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/LARC.py -> build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/distributed.py -> build/lib.linux-x86_64-3.6/apex/parallel
copying apex/parallel/optimized_sync_batchnorm_kernel.py -> build/lib.linux-x86_64-3.6/apex/parallel
creating build/lib.linux-x86_64-3.6/apex/reparameterization
copying apex/reparameterization/weight_norm.py -> build/lib.linux-x86_64-3.6/apex/reparameterization
copying apex/reparameterization/__init__.py -> build/lib.linux-x86_64-3.6/apex/reparameterization
copying apex/reparameterization/reparameterization.py -> build/lib.linux-x86_64-3.6/apex/reparameterization
creating build/lib.linux-x86_64-3.6/apex/fp16_utils
copying apex/fp16_utils/fp16_optimizer.py -> build/lib.linux-x86_64-3.6/apex/fp16_utils
copying apex/fp16_utils/fp16util.py -> build/lib.linux-x86_64-3.6/apex/fp16_utils
copying apex/fp16_utils/__init__.py -> build/lib.linux-x86_64-3.6/apex/fp16_utils
copying apex/fp16_utils/loss_scaler.py -> build/lib.linux-x86_64-3.6/apex/fp16_utils
creating build/lib.linux-x86_64-3.6/apex/optimizers
copying apex/optimizers/fp16_optimizer.py -> build/lib.linux-x86_64-3.6/apex/optimizers
copying apex/optimizers/__init__.py -> build/lib.linux-x86_64-3.6/apex/optimizers
copying apex/optimizers/fused_adam.py -> build/lib.linux-x86_64-3.6/apex/optimizers
creating build/lib.linux-x86_64-3.6/apex/amp/lists
copying apex/amp/lists/torch_overrides.py -> build/lib.linux-x86_64-3.6/apex/amp/lists
copying apex/amp/lists/tensor_overrides.py -> build/lib.linux-x86_64-3.6/apex/amp/lists
copying apex/amp/lists/__init__.py -> build/lib.linux-x86_64-3.6/apex/amp/lists
copying apex/amp/lists/functional_overrides.py -> build/lib.linux-x86_64-3.6/apex/amp/lists
running build_ext
building 'apex_C' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/csrc
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/torch/csrc/api/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/TH -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/THC -I/usr/include/python3.6m -c csrc/flatten_unflatten.cpp -o build/temp.linux-x86_64-3.6/csrc/flatten_unflatten.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=apex_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/csrc/flatten_unflatten.o -o build/lib.linux-x86_64-3.6/apex_C.cpython-36m-x86_64-linux-gnu.so
building 'amp_C' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/torch/csrc/api/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/TH -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c csrc/amp_C_frontend.cpp -o build/temp.linux-x86_64-3.6/csrc/amp_C_frontend.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
/usr/local/cuda/bin/nvcc -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/torch/csrc/api/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/TH -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c csrc/multi_tensor_scale_kernel.cu -o build/temp.linux-x86_64-3.6/csrc/multi_tensor_scale_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -lineinfo -O3 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/usr/local/cuda/bin/nvcc -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/torch/csrc/api/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/TH -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c csrc/multi_tensor_axpby_kernel.cu -o build/temp.linux-x86_64-3.6/csrc/multi_tensor_axpby_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -lineinfo -O3 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/usr/local/cuda/bin/nvcc -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/torch/csrc/api/include -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/TH -I/home/lab324/.local/lib/python3.6/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c csrc/multi_tensor_l2norm_kernel.cu -o build/temp.linux-x86_64-3.6/csrc/multi_tensor_l2norm_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -lineinfo -O3 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
csrc/type_shim.h(118): error: identifier "__shfl_down_sync" is undefined
          detected during instantiation of "T reduce_block_into_lanes(T *, T, int, __nv_bool) [with T=float]"
csrc/multi_tensor_l2norm_kernel.cu(67): here

1 error detected in the compilation of "/tmp/tmpxft_00004094_00000000-7_multi_tensor_l2norm_kernel.cpp1.ii".
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 2

Running setup.py install for apex ... error
Cleaning up...
Removing source in /tmp/pip-req-build-i31tt77s
Removed build tracker '/tmp/pip-req-tracker-b7tfd9v5'
ERROR: Command "/usr/bin/python3 -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-req-build-i31tt77s/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' --cpp_ext --cuda_ext install --record /tmp/pip-record-pc5mw3el/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-req-build-i31tt77s/
Exception information:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/base_command.py", line 178, in main
status = self.run(options, args)
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py", line 414, in run
use_user_site=options.use_user_site,
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/init.py", line 58, in install_given_reqs
**kwargs
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/req_install.py", line 951, in install
spinner=spinner,
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/misc.py", line 776, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip._internal.exceptions.InstallationError: Command "/usr/bin/python3 -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-req-build-i31tt77s/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' --cpp_ext --cuda_ext install --record /tmp/pip-record-pc5mw3el/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-req-build-i31tt77s/
1 location(s) to search for versions of pip:

I can't figure it out. Could you help me? Thank you! @mcarilli

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions