when install from sources, I encounter ERROR: C++ compilation of rule '//google/protobuf:protobuf_lite' failed: crosstool_wrapper_driver_is_not_gcc failed

[xuezhisd]

Do anyone compile the source of Tensor Flow?
When I install Tensor Flow from sources, After I excute under command in terminal.

bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer

I encounter the follow error.

ERROR: ~/tensorflow/google/protobuf/BUILD:29:1: C++ compilation of rule 
'//google/protobuf:protobuf_lite' failed: crosstool_wrapper_driver_is_not_gcc failed: error 
executing command third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -
U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-
parameter -Wno-free-nonheap-object ... (remaining 40 argument(s) skipped).
Target //tensorflow/cc:tutorials_example_trainer failed to build

My OS is Ubuntu 14.04，GCC/G++ is 4.7, GPU is K40c, Python is 2.7, cuda is 7.0, cudnn is 6.5v2, bazel is installed by compile source code.

[ebrevdo]

Can you run the build with the flag --verbose_failures? Perhaps this will give more info.

[xuezhisd]

Yes,no Problem.
I run the build with the flag --verbose_failures by excute under code in terminal.

bazel build -c opt --config=cuda --verbose_failures //tensorflow/cc:tutorials_example_trainer

And, follow is my ERROR information

# command
root@optimal4:/home/share/TensorFlow/tensorflow# bazel build -c opt --config=cuda --
verbose_failures //tensorflow/cc:tutorials_example_trainer

# ERROR information
INFO: Found 1 target...
INFO: From Compiling google/protobuf/src/google/protobuf/compiler/objectivec/objectivec_oneof.cc 
[for host]:
src/main/tools/namespace-sandbox.c:633: execvp(argv[0], argv): No such file or directory
ERROR: /home/share/TensorFlow/tensorflow/google/protobuf/BUILD:166:1: C++ compilation of rule 
'//google/protobuf:protoc_lib' failed: crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /root/.cache/bazel/_bazel_root/def3d5a3fd5ec6e6da9184558cd2b5fe/tensorflow && \
  exec env - \
    PATH=.:/usr/local/jdk/bin:/usr/local/jdk/jre/bin:/home/share/TensorFlow/bazel/output:/home/xuezhisd/w
ork/caffe/build/tools:/usr/local/ffmpeg/bin:/usr/local/cuda/bin:usr/local/MATLAB/R2014b/bin:.:/usr/local/j
dk/bin:/usr/local/jdk/jre/bin:/usr/local/cuda/bin:/usr/local/MATLAB/R2014b/bin:/usr/local/sbin:/usr/local/b
in:/usr/sbin:/usr/bin:/sbin:/bin \
  third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-
D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-
nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections -g0 '-
std=c++11' -iquote . -iquote bazel-out/host/genfiles -iquote external/bazel_tools -iquote bazel-
out/host/genfiles/external/bazel_tools -isystem google/protobuf/src -isystem bazel-
out/host/genfiles/google/protobuf/src -isystem external/bazel_tools/tools/cpp/gcc3 -DHAVE_PTHREAD 
-Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare '-Wno-error=unused-function' -no-
canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-
D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-frandom-seed=bazel-
out/host/bin/google/protobuf/_objs/protoc_lib/google/protobuf/src/google/protobuf/compiler/objectivec/
objectivec_oneof.o' -MD -MF bazel-
out/host/bin/google/protobuf/_objs/protoc_lib/google/protobuf/src/google/protobuf/compiler/objectivec/
objectivec_oneof.d -c google/protobuf/src/google/protobuf/compiler/objectivec/objectivec_oneof.cc -o 
bazel-
out/host/bin/google/protobuf/_objs/protoc_lib/google/protobuf/src/google/protobuf/compiler/objectivec/
objectivec_oneof.o): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /root/.cache/bazel/_bazel_root/def3d5a3fd5ec6e6da9184558cd2b5fe/tensorflow && \
  exec env - \
    PATH=.:/usr/local/jdk/bin:/usr/local/jdk/jre/bin:/home/share/TensorFlow/bazel/output:/home/xuezhisd/w
ork/caffe/build/tools:/usr/local/ffmpeg/bin:/usr/local/cuda/bin:usr/local/MATLAB/R2014b/bin:.:/usr/local/j
dk/bin:/usr/local/jdk/jre/bin:/usr/local/cuda/bin:/usr/local/MATLAB/R2014b/bin:/usr/local/sbin:/usr/local/b
in:/usr/sbin:/usr/bin:/sbin:/bin \
  third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-
D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-
nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections -g0 '-
std=c++11' -iquote . -iquote bazel-out/host/genfiles -iquote external/bazel_tools -iquote bazel-
out/host/genfiles/external/bazel_tools -isystem google/protobuf/src -isystem bazel-
out/host/genfiles/google/protobuf/src -isystem external/bazel_tools/tools/cpp/gcc3 -DHAVE_PTHREAD 
-Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare '-Wno-error=unused-function' -no-
canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-
D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-frandom-seed=bazel-
out/host/bin/google/protobuf/_objs/protoc_lib/google/protobuf/src/google/protobuf/compiler/objectivec/
objectivec_oneof.o' -MD -MF bazel-
out/host/bin/google/protobuf/_objs/protoc_lib/google/protobuf/src/google/protobuf/compiler/objectivec/
objectivec_oneof.d -c google/protobuf/src/google/protobuf/compiler/objectivec/objectivec_oneof.cc -o 
bazel-
out/host/bin/google/protobuf/_objs/protoc_lib/google/protobuf/src/google/protobuf/compiler/objectivec/
objectivec_oneof.o).
Target //tensorflow/cc:tutorials_example_trainer failed to build
INFO: Elapsed time: 2.362s, Critical Path: 1.03s.

[SWu]

This happened to me when I tried to use a different version of bazel than 0.1.0

[ebrevdo]

Can you say what version of bazel you are using? And try @SWu's advice and see if it solves the problem?

[xuezhisd]

I used bazel-master.

When I try to compile and install bazel 0.1.0, I also encounter some errors.
I excute follow command,

[root@/home/share/TensorFlow/bazel-0.1.0]#./compile.sh build --verbose_failures

and I got follow error information.

INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    ./compile.sh build /path/to/bazel
🍃  Building Bazel from scratch............
🍃  Building Bazel with Bazel.
.Extracting Bazel installation...
Sending SIGTERM to previous Bazel server (pid=42474)... done.
........
INFO: Found 1 target...
INFO: From Linking src/main/native/libunix.so:
/usr/bin/ld: bazel-out/local_linux-
fastbuild/bin/src/main/native/_objs/libunix.so/src/main/native/unix_jni.pic.o: relocation 
R_X86_64_PC32 against symbol `_Z13PostExceptionP7JNIEnv_iRKSs' can not be used
 when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
ERROR: /home/share/TensorFlow/bazel-0.1.0/src/main/native/BUILD:28:1: Linking of rule 
'//src/main/native:libunix.so' failed: gcc failed: error executing command /usr/bin/gcc 
-shared -o bazel-out/local_linux-fastbuild/bin/src/main/native/libunix.so -Wl,-whole-archive ... 
(remaining 16 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: 
Process exited with status 1.
ERROR: Sandboxed execution failed: //src/java_tools/buildjar:skylark-deps..
ERROR: Sandboxed execution failed: //src/main/protobuf:proto_crosstool_config_srcjar..
ERROR: Sandboxed execution failed: //src:android_tools_repository_zip..
ERROR: Sandboxed execution failed: //src/main/java:shell-skylark..
Target //src:bazel failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 6.436s, Critical Path: 2.87s

[ebrevdo]

Are you on a 32 bit system?
On Nov 13, 2015 5:40 PM, "xuezhisd" notifications@github.com wrote:

I used bazel-master.

When I try to compile and install bazel 0.1.0, I also encounter some
errors.
I excute follow command,

[root@/home/share/TensorFlow/bazel-0.1.0]#./compile.sh build --verbose_failures

and I got follow error information.

INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO: ./compile.sh build /path/to/bazel
🍃 Building Bazel from scratch............
🍃 Building Bazel with Bazel.
.Extracting Bazel installation...
Sending SIGTERM to previous Bazel server (pid=42474)... done.
........
INFO: Found 1 target...
INFO: From Linking src/main/native/libunix.so:
/usr/bin/ld: bazel-out/local_linux-
fastbuild/bin/src/main/native/_objs/libunix.so/src/main/native/unix_jni.pic.o: relocation
R_X86_64_PC32 against symbol `_Z13PostExceptionP7JNIEnv_iRKSs' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
ERROR: /home/share/TensorFlow/bazel-0.1.0/src/main/native/BUILD:28:1: Linking of rule
'//src/main/native:libunix.so' failed: gcc failed: error executing command /usr/bin/gcc
-shared -o bazel-out/local_linux-fastbuild/bin/src/main/native/libunix.so -Wl,-whole-archive ...
(remaining 16 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException:
Process exited with status 1.
ERROR: Sandboxed execution failed: //src/java_tools/buildjar:skylark-deps..
ERROR: Sandboxed execution failed: //src/main/protobuf:proto_crosstool_config_srcjar..
ERROR: Sandboxed execution failed: //src:android_tools_repository_zip..
ERROR: Sandboxed execution failed: //src/main/java:shell-skylark..
Target //src:bazel failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 6.436s, Critical Path: 2.87s

—
Reply to this email directly or view it on GitHub
#190 (comment)
.

[xuezhisd]

no, my system is Ubuntu Server 14.04 64bit

[root@/home/share/backup/tensorflow]#uname -a
Linux optimal4 3.16.0-49-generic #65~14.04.1-Ubuntu SMP Wed Sep 9 10:03:23 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

[ebrevdo]

This issue is tracked by Bazel here where they offer two possible solutions.

[XingxingZhang]

Hi @ebrevdo

Background:
I am installing tensorflow from source, since I am using a server where I don't have the root access.
The default python version on that server is python 2.6 (but I have my own virtual environment with python 2.7).
The default gcc is "4.4.7", but we also have a gcc "4.9.1" somewhere else.
I have successfully installed bazel.

Problems:
The bazel issue you mentioned has already been fixed in later version.
But I still have a similar bug and here is the error message "third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc".

Besides, I have configured tensorflow to use python2.7 in tensorflow's ./configure
Why tf get " from argparse import ArgumentParser
ImportError: No module named argparse"?
Clearly, tf is using the default python 2.6 on our server

$ ../bazel/output/bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer --verbose_failures
INFO: Waiting for response from Bazel server (pid 6348)...
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.io/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing.
INFO: Found 1 target...
ERROR: /afs/inf.ed.ac.uk/user/s12/s1270921/.cache/bazel/_bazel_s1270921/84c5f700ae424d93a6550e3150808411/external/protobuf/BUILD:71:1: C++ compilation of rule '@protobuf//:protobuf_lite' failed: crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /afs/inf.ed.ac.uk/user/s12/s1270921/.cache/bazel/_bazel_s1270921/84c5f700ae424d93a6550e3150808411/execroot/tensorflow && \
  exec env - \
    LD_LIBRARY_PATH=/afs/inf.ed.ac.uk/user/s12/s1270921/torch_cudnn/install/lib:/afs/inf.ed.ac.uk/user/s12/s1270921/torch/install/lib:/afs/inf.ed.ac.uk/user/s12/s1270921/torch_cudnn/install/lib:/afs/inf.ed.ac.uk/user/s12/s1270921/torch/install/lib::/afs/inf.ed.ac.uk/user/s12/s1270921/usr/kenlm/lib:/afs/inf.ed.ac.uk/group/project/img2txt/soft/usr/lib:/usr/lib64/atlas:/afs/inf.ed.ac.uk/user/s12/s1270921/usr/kenlm/lib:/afs/inf.ed.ac.uk/group/project/img2txt/soft/usr/lib:/usr/lib64/atlas \
    PATH=/afs/inf.ed.ac.uk/user/s12/s1270921/torch_cudnn/install/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/torch/install/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/torch_cudnn/install/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/torch/install/bin:/home/s1270921/bin/:/usr/local/bin/:/opt/matlab-R2015a/bin/:/usr/local/sbin:/usr/bin:/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/bin:/afs/inf.ed.ac.uk/group/project/img2txt/soft/usr/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/irstlm/bin:/home/52nlp/mtworkdir/irstlm/bin://home/52nlp/mtworkdir/irstlm/bin/x86_64-redhat-linux-gnu:/afs/inf.ed.ac.uk/user/s12/s1270921/SRILM/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/SRILM/bin/i686-m64:/afs/inf.ed.ac.uk/user/s12/s1270921/.local/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/bin:/afs/inf.ed.ac.uk/group/project/img2txt/soft/usr/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/irstlm/bin:/home/52nlp/mtworkdir/irstlm/bin://home/52nlp/mtworkdir/irstlm/bin/x86_64-redhat-linux-gnu:/afs/inf.ed.ac.uk/user/s12/s1270921/SRILM/bin:/afs/inf.ed.ac.uk/user/s12/s1270921/SRILM/bin/i686-m64:/afs/inf.ed.ac.uk/user/s12/s1270921/.local/bin \
  third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections -g0 '-std=c++11' -MD -MF bazel-out/host/bin/external/protobuf/_objs/protobuf_lite/external/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.d '-frandom-seed=bazel-out/host/bin/external/protobuf/_objs/protobuf_lite/external/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.o' -iquote external/protobuf -iquote bazel-out/host/genfiles/external/protobuf -iquote external/bazel_tools -iquote bazel-out/host/genfiles/external/bazel_tools -isystem external/protobuf/src -isystem bazel-out/host/genfiles/external/protobuf/src -isystem external/bazel_tools/tools/cpp/gcc3 -DHAVE_PTHREAD -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare '-Wno-error=unused-function' -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fno-canonical-system-headers -c external/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.cc -o bazel-out/host/bin/external/protobuf/_objs/protobuf_lite/external/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.o): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Traceback (most recent call last):
  File "third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc", line 41, in <module>
    from argparse import ArgumentParser
ImportError: No module named argparse
Target //tensorflow/cc:tutorials_example_trainer failed to build
INFO: Elapsed time: 8.452s, Critical Path: 2.73s

[zhangwei0119]

same problem here.

[abewley]

I'm in the same situation as well. Did you find a solution for getting it to use Python 2.7 instead of the server default version?

[XingxingZhang]

@abewley

No, I didn't find a solution. In the end, I asked the administrator to upgrade our gcc, python and other related dependencies.

[diggerdu]

same problem here.