Getting "missing dependency declarations" with bazel 0.3.0

[yaroslavvb]

I just upgraded Bazel/synced and now I'm getting same errors as in #1157

bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer

ERROR: /home/yaroslavvb/tensorflow.git/tensorflow/tensorflow/core/kernels/BUILD:1080:1: undeclared inclusion(s) in rule '//tensorflow/core/kernels:cwise_op_gpu':
this rule is missing dependency declarations for the following files included by 'tensorflow/core/kernels/cwise_op_gpu_floor.cu.cc':
  '/usr/local/cuda-8.0/include/cuda_runtime.h'
  '/usr/local/cuda-8.0/include/host_config.h'
  '/usr/local/cuda-8.0/include/builtin_types.h'
  '/usr/local/cuda-8.0/include/device_types.h'
  '/usr/local/cuda-8.0/include/host_defines.h'
  '/usr/local/cuda-8.0/include/driver_types.h'
  '/usr/local/cuda-8.0/include/surface_types.h'
  '/usr/local/cuda-8.0/include/texture_types.h'
...

[yaroslavvb]

Work-around from #1157 seems to work

add
cxx_builtin_include_directory: "/usr/local/cuda-8.0/include"
to
third_party/gpus/crosstool/CROSSTOOL

[concretevitamin]

Good to know it can be made to work, @yaroslavvb thanks for persisting the workaround.

[yaroslavvb]

Weird that you guys aren't hitting this issue on the nightly wheels, unless the nightlies are done with newer Bazel than 0.3.0

[AdamBear]

Same "missing dependency declarations" problem still exists in head version, thanks @yaroslavvb for the workaround.

[mschonwe]

Just updated to current version cc3153a (v0.10.0rc0-785-gcc3153a) and got the usual missing dependency error.

HOWEVER, the new build doesn't create a CROSSTOOL file - so the usual fix doesn't work... I tried copying in the CROSSTOOL from an older revision (with the cxx_builtin_include_directory: "/usr/local/cuda-8.0/include" change) - but I'm still getting the missing dependency error.
Not sure what .tpl files are, but I tried adding the cxx_builtin change in that file, also no luck.

Installed version of CUDA and cuDNN: 8.0.26 and v5
bazel version : 0.3.0
Ubuntu 14.04.4 LTS
3xGTX980Ti

UPDATE:
In case this helps anyone... I was able to get past the issue by setting the CUDA version explicitly when running the configure script.

[elexira]

guys i am trying to

add
cxx_builtin_include_directory: "/usr/local/cuda-8.0/include"
to
third_party/gpus/crosstool/CROSSTOOL

but i get the following, where in the CROSSTOOL should i copy paste that line ?

ERROR: java.io.IOException: Could not read the crosstool configuration file 'CROSSTOOL file /home/---/Downloads/tensorflow/third_party/gpus/crosstool/CROSSTOOL', because of a parser error (1:1: Input contains unknown fields and/or extensions:
1:1: com.google.devtools.build.lib.view.config.crosstool.CrosstoolRelease.cxx_builtin_include_directory).

[jiapei100]

@Sadrpour

Exactly the same error here ... I'm testing on the most current bazel-git, namely, bazel 0.3.2
https://github.com/bazelbuild/bazel

The error message is:

ERROR: java.io.IOException: Could not read the crosstool configuration file 'CROSSTOOL file /tmp/bazel_uawUdYmF/out/external/local_config_cc/CROSSTOOL', because of a parser error (132:42: String missing ending quote.).

cheers
Pei

[dzhyeon]

@Sadrpour @jiapei100 I guess the parsing error was caused because you put the line in a wrong location.

putting it like the below will cause an parsing error.
default_target_cpu: "same_as_host"
cxx_builtin_include_directory: "/usr/local/cuda-8.0/include"

putting it inside toolchain namespace like:
toolchain{
cxx_builtin_include_directory: "/usr/local/cuda-8.0/include"
}

caused no problem, but still it is causing missing dependency problem.

[jiapei100]

@dzhyeon
I actually posted a very similar issue at bazelbuild/bazel#1996 .

Which file and which line should this line
cxx_builtin_include_directory: "/usr/local/cuda-8.0/include
be added to?

Cheers
Pei

[irfan-zoefit]

Getting following error after updating cxx_builtin_include_directory: "/usr/local/cuda-7.5/include

bazel test -c opt --config=cuda --define using_cuda_nvcc=true --define using_gcudacc=true syntaxnet/... util/utf8/...
................
WARNING: /home/irfan/.cache/bazel/_bazel_irfan/a05fc8a5ac651b688321e83d1f272360/external/org_tensorflow/tensorflow/workspace.bzl:72:5: tf_repo_name was specified to tf_workspace but is no longer used and will be removed in the future.
ERROR: java.io.IOException: Could not read the crosstool configuration file 'CROSSTOOL file /home/irfan/.cache/bazel/_bazel_irfan/a05fc8a5ac651b688321e83d1f272360/external/local_config_cuda/crosstool/CROSSTOOL', because of a parser error (259:1: Input contains unknown fields and/or extensions:
259:1:	com.google.devtools.build.lib.view.config.crosstool.CrosstoolRelease.cxx_builtin_include_directory).
INFO: Elapsed time: 6.346s
ERROR: Couldn't start the build. Unable to run tests.

[BruceDai003]

Just updated to current version cc3153a (v0.10.0rc0-785-gcc3153a) and got the usual missing dependency error.

HOWEVER, the new build doesn't create a CROSSTOOL file - so the usual fix doesn't work... I tried copying in the CROSSTOOL from an older revision (with the cxx_builtin_include_directory: "/usr/local/cuda-8.0/include" change) - but I'm still getting the missing dependency error.
Not sure what .tpl files are, but I tried adding the cxx_builtin change in that file, also no luck.

Installed version of CUDA and cuDNN: 8.0.26 and v5
bazel version : 0.3.0
Ubuntu 14.04.4 LTS
3xGTX980Ti

UPDATE:
In case this helps anyone... I was able to get past the issue by setting the CUDA version explicitly when running the configure script.

How to set CUDA version explicitly?