Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build from source -> build the pip package -> GPU support -> bazel build -> ERROR: Config value cuda is not defined in any .rc file #23401

Closed
gopi77 opened this issue Oct 31, 2018 · 12 comments

Comments

@gopi77
Copy link

gopi77 commented Oct 31, 2018

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
  • TensorFlow installed from (source or binary): Source
  • TensorFlow version: v1.12.0-rc2
  • Python version: Python 3.7.0
  • Installed using virtualenv? pip? conda?: conda 4.5.11
  • Bazel version (if compiling from source): 0.19.0
  • GCC/Compiler version (if compiling from source): GCC 7.2.0
  • CUDA/cuDNN version: 10.0 / 7.3.1
  • GPU model and memory: nvidia GTX 1070 / 8GB

Describe the problem
Got error while building pip package from source
ERROR: Config value cuda is not defined in any .rc file

Provide the exact sequence of commands / steps that you executed before running into the problem
gopi@gp:~/tensorflow$ ./configure
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/home/gopi/tensorflow/tools/bazel.rc
WARNING: Running Bazel server needs to be killed, because the startup options are different.
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.19.0 installed.
Please specify the location of python. [Default is /home/gopi/anaconda3/bin/python]:

Found possible Python library paths:
/home/gopi/anaconda3/lib/python3.7/site-packages
Please input the desired Python library path to use. Default is [/home/gopi/anaconda3/lib/python3.7/site-packages]

Do you wish to build TensorFlow with Apache Ignite support? [Y/n]: n
No Apache Ignite support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: N
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: N
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 9.0]: 10.0

Please specify the location where CUDA 10.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]:

Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Do you wish to build TensorFlow with TensorRT support? [y/N]: N
No TensorRT support will be enabled for TensorFlow.

Please specify the NCCL version you want to use. If NCCL 2.2 is not installed, then you can use version 1.3 that can be fetched automatically but it may have worse performance with multiple GPUs. [Default is 2.2]: 1.3

Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 6.1]:

Do you want to use clang as CUDA compiler? [y/N]: N
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:

Do you wish to build TensorFlow with MPI support? [y/N]: N
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: N
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=gdr # Build with GDR support.
--config=verbs # Build with libverbs support.
--config=ngraph # Build with Intel nGraph support.
Configuration finished
gopi@gp:~/tensorflow$ bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/home/gopi/tensorflow/tools/bazel.rc
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=204
INFO: Reading rc options for 'build' from /home/gopi/tensorflow/.tf_configure.bazelrc:
'build' options: --action_env PYTHON_BIN_PATH=/home/gopi/anaconda3/bin/python --action_env PYTHON_LIB_PATH=/home/gopi/anaconda3/lib/python3.7/site-packages --python_path=/home/gopi/anaconda3/bin/python --action_env TF_NEED_OPENCL_SYCL=0 --action_env TF_NEED_ROCM=0 --action_env TF_NEED_CUDA=1 --action_env CUDA_TOOLKIT_PATH=/usr/local/cuda --action_env TF_CUDA_VERSION=10.0 --action_env CUDNN_INSTALL_PATH=/usr/local/cuda-10.0 --action_env TF_CUDNN_VERSION=7 --action_env TF_NCCL_VERSION=1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=6.1 --action_env LD_LIBRARY_PATH=:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64 --action_env TF_CUDA_CLANG=0 --action_env GCC_HOST_COMPILER_PATH=/usr/bin/gcc --config=cuda
ERROR: Config value cuda is not defined in any .rc file

Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

@wdirons
Copy link
Contributor

wdirons commented Oct 31, 2018

This is an issue with bazel 0.19.0, either use bazel 0.18.0, or follow the warning message:

WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/home/gopi/tensorflow/tools/bazel.rc

and add the contents of that file to your .tf_configure.bazelrc

@gopi77
Copy link
Author

gopi77 commented Oct 31, 2018

Thanks William :)
Using the same version of bazel 0.19.0, Added the content of file "/home/gopi/tensorflow/tools/bazel.rc" on top of (hidden) file "/home/gopi/tensorflow/.tf_configure.bazelrc" & build happens.

@hstart
Copy link

hstart commented Nov 5, 2018

build tensorflow 1.11.0 I have do this,in source file add context "
import /home/ai/ver/tensorflow-r1.11/tools/bazel.rc " on top line of(hide file) "/home/ai/ver/tensorflow-r1.11/.bazelrc " and not .tf_configure.bazelrc

@shahruk10
Copy link

build tensorflow 1.11.0 I have do this,in source file add context "
import /home/ai/ver/tensorflow-r1.11/tools/bazel.rc " on top line of(hide file) "/home/ai/ver/tensorflow-r1.11/.bazelrc " and not .tf_configure.bazelrc

This worked for me too. Thanks !

@LyonOconner
Copy link

LyonOconner commented Dec 11, 2018

tensorflow

how is this ? A example pls ....

I found TensorFlow\models\research\syntaxnet\tools\bazel.rc and now ??

@s1113950
Copy link

s1113950 commented Jan 9, 2019

I had an issue with version 0.20.0 as well with building with mkl support. I downgraded to 0.18.1 and all was well.

Bad version:

[root@crobiso1-container tensorflow]# bazel version
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/tensorflow/tools/bazel.rc
WARNING: detected http_proxy set in env, setting no_proxy for localhost.
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
INFO: Invocation ID: 4ca86d39-5a93-4068-8467-4f4298a986c8
Build label: 0.20.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Nov 30 14:39:01 2018 (1543588741)
Build timestamp: 1543588741
Build timestamp as int: 1543588741

Failing build:

[root@crobiso1-container tensorflow]# echo $TF_BAZEL_BUILD_OPTIONS
--config=mkl --cxxopt=-D_GLIBCXX_USE_CXX11_ABI=0 --copt=-march=corei7-avx --copt=-mtune=core-avx-i --copt=-O3
[root@crobiso1-container tensorflow]# bazel build -c opt \
>         ${TF_BAZEL_BUILD_OPTIONS} \
>         tensorflow/tools/pip_package:build_pip_package
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/tensorflow/tools/bazel.rc
WARNING: detected http_proxy set in env, setting no_proxy for localhost.
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=189
INFO: Reading rc options for 'build' from /etc/bazel.bazelrc:
  'build' options: --spawn_strategy=standalone --genrule_strategy=standalone
INFO: Reading rc options for 'build' from /tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/lib/python3.4/site-packages --python_path=/usr/bin/python3 --define with_ignite_support=true --define with_xla_support=true --action_env TF_NEED_OPENCL_SYCL=0 --action_env TF_NEED_ROCM=0 --action_env TF_NEED_CUDA=0 --action_env TF_DOWNLOAD_CLANG=0
ERROR: Config value mkl is not defined in any .rc file
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
INFO: Invocation ID: 5ab95a91-820d-4262-9a8a-1c6312316ba0
[root@crobiso1-container tensorflow]# echo $?
2

@SimonBoorer
Copy link

SimonBoorer commented Jan 22, 2019

This was broken by this commit:
#22964

PR raised to fix it here:
#25114

You'll probably need to cherry-pick this commit if you are compiling with cuda support:
09feb69

@bzamecnik
Copy link
Contributor

Or edit the .bazel.rc file in-place with sed:

sed -i -E 's#\.tf_configure\.bazelrc#tools/bazel.rc#' .bazelrc

@mrgloom
Copy link

mrgloom commented Mar 19, 2019

I can't find .tf_configure.bazelrc when it should appear?

@wdirons
Copy link
Contributor

wdirons commented Mar 19, 2019

@mrgloom, .tf_configure.bazelrc is created by running ./configure

Your answers from ./configure are stored in .tf_configure.bazelrc

@leonidas-timefly
Copy link

Thanks William :)
Using the same version of bazel 0.19.0, Added the content of file "/home/gopi/tensorflow/tools/bazel.rc" on top of (hidden) file "/home/gopi/tensorflow/.tf_configure.bazelrc" & build happens.

how can I find hidden file .tf_configure.bazelrc?

@leonidas-timefly
Copy link

Thanks William :)
Using the same version of bazel 0.19.0, Added the content of file "/home/gopi/tensorflow/tools/bazel.rc" on top of (hidden) file "/home/gopi/tensorflow/.tf_configure.bazelrc" & build happens.

OK,I' known,use “ls -a” to see hidden file ,thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants