error: namespace "Eigen::half_impl" has no member "__half_raw" when building latest TensorFlow/CUDA 9.0

[yaroslavvb]

When trying to build latest TensorFlow for CUDA 9.0/CuDNN 7.0, from today's head

Complete instructions to reproduce are here, but basically I followed steps in in Dockerfile.devel-gpu to install dependencies, and then did bazel build -c opt --config=cuda

Here are the errors

INFO: From Compiling tensorflow/stream_executor/cuda/cuda_blas.cc:
tensorflow/stream_executor/cuda/cuda_blas.cc: In function 'cudaDataType_t perftools::gputools::cuda::{anonymous}::CUDAComputationType(perftools::gputools::blas::ComputationType)':
tensorflow/stream_executor/cuda/cuda_blas.cc:527:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
INFO: From Compiling tensorflow/core/kernels/dense_to_sparse_batch_dataset_op.cc:
tensorflow/core/kernels/dense_to_sparse_batch_dataset_op.cc: In member function 'virtual void tensorflow::{anonymous}::DenseToSparseBatchDatasetOp::MakeDataset(tensorflow::OpKernelContext*, tensorflow::DatasetBase*, tensorflow::DatasetBase**)':
tensorflow/core/kernels/dense_to_sparse_batch_dataset_op.cc:83:71: warning: 'batch_size' may be used uninitialized in this function [-Wmaybe-uninitialized]
         : batch_size_(batch_size), row_shape_(row_shape), input_(input) {
                                                                       ^
tensorflow/core/kernels/dense_to_sparse_batch_dataset_op.cc:41:11: note: 'batch_size' was declared here
     int64 batch_size;
           ^
INFO: From Compiling tensorflow/core/kernels/histogram_op_gpu.cu.cc:
./tensorflow/core/util/cuda_kernel_helper.h(109): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(109): error: expected a ")"

./tensorflow/core/util/cuda_kernel_helper.h(110): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(113): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(113): error: expected a ";"

./tensorflow/core/util/cuda_kernel_helper.h(119): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(119): error: expected a ")"

./tensorflow/core/util/cuda_kernel_helper.h(120): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(123): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(123): error: expected a ";"

10 errors detected in the compilation of "/tmp/tmpxft_000137d9_00000000-6_histog

[gunan]

#14770 should fix this issue.

[yzhang092]

#14770 seems not fix this issue. Still get error from cuda_kernel_helper

[nevakanezzar]

seconded, same issue.

[maxiaodong97]

looking into the history the breaking code introduced yesterday by
Wrappers for CUDA 9 warp-synchronous intrinsics.

[yaroslavvb]

Cc @zheng-xq

[gunan]

Looks like I was wrong, the PR I mentioned did not fix this completely. However this is fixed internally.
@caisq when are we planning the next push from internal to github?

[caisq]

I'll push as soon as the ongoing pull is complete, @gunan

[JackShen2015]

I tried to build it, but it seems that the issue still exist

[castleguarders]

FYI, Issue still there in case the push already happened.

[caisq]

The push is open in PR #15174, awaiting approval.

[gunan]

This should now be fixed.

[samikama]

This is back again. Commit 1ad680d seems to break it again.

[David-Levinthal]

just leaving some of the build error messages from a cuda9.1 build with openmpi
INFO: From Compiling tensorflow/core/kernels/gather_functor_gpu.cu.cc:^M
./tensorflow/core/util/cuda_kernel_helper.h(116): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(116): error: expected a ")"

./tensorflow/core/util/cuda_kernel_helper.h(117): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(120): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(120): error: expected a ";"

./tensorflow/core/util/cuda_kernel_helper.h(126): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(126): error: expected a ")"

./tensorflow/core/util/cuda_kernel_helper.h(127): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(130): error: namespace "Eigen::half_impl" has no member "__half_raw"

./tensorflow/core/util/cuda_kernel_helper.h(130): error: expected a ";"

10 errors detected in the compilation of "/tmp/tmpxft_00004514_00000000-6_gather_functor_gpu.cu.cpp1.ii".
ERROR: /home/levinth/tensorflow/tensorflow/core/kernels/BUILD:1139:1: output 'tensorflow/core/kernels/_objs/gather_functor_gpu/tensorflow/core/kernels/gather_functor_gpu.cu.pic.o' was not created

[gunan]

@reedwm @zhengxq FYI.

I think we have an internal fix, we are waiting for that to be pushed out.
@reedwm, could you confirm?

[reedwm]

1ad680d has been rolled back internally, so this should be fixed internally. Thank you @samikama for identifying the specific commit that broke it.

[David-Levinthal]

when will we see the fix externally?

[gunan]

Should be at master now.

…

On Dec 23, 2017 9:26 AM, "David Levinthal Ph.D." ***@***.***> wrote: when will we see the fix externally? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#15108 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHlCOSprlp48-GpM_vnx1dbV9hLSnZ5qks5tDTfZgaJpZM4Q1e9q> .

[David-Levinthal]

yup..fresh git clone and INFO: Elapsed time: 362.184s, Critical Path: 157.04s^M INFO: Build completed successfully, 2767 total actions^M On Sat, Dec 23, 2017 at 11:56 AM, Gunhan Gulsoy <notifications@github.com> wrote:

…

Should be at master now. On Dec 23, 2017 9:26 AM, "David Levinthal Ph.D." ***@***.*** > wrote: > when will we see the fix externally? > > — > You are receiving this because you modified the open/close state. > Reply to this email directly, view it on GitHub > <https://github.com/tensorflow/tensorflow/issues/ 15108#issuecomment-353738119>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AHlCOSprlp48-GpM_ vnx1dbV9hLSnZ5qks5tDTfZgaJpZM4Q1e9q> > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#15108 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIUuT0U7Mwnejouvfo7Jk0_ZkPFbThrCks5tDVsBgaJpZM4Q1e9q> .

[gunan]

Thanks for verifying!
Then I am closing this issue.