CUDA compiling include file path failure inside tensorflow python installation directory in Windows 10

[k-l-lambda]

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): v2.2.0-rc4-8-g2b96f3662b 2.2.0
• Python version: 3.6
• Bazel version (if compiling from source): N/A
• GCC/Compiler version (if compiling from source): N/A
• CUDA/cuDNN version: 10.1
• GPU model and memory: (not related with this issue)

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:

1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior

Create a CUDA source file, e.g. named test.cu, with content:

#define EIGEN_USE_GPU
#define __CUDA_INCLUDE_COMPILER_INTERNAL_HEADERS__
#include "tensorflow/core/framework/op_kernel.h"

Then run command:
nvcc ./test.cu --include-path "C:\Users\xxx\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\include"

Then you will got error like this:

C:/Users/xxx/AppData/Local/Programs/Python/Python36/lib/site-packages/tensorflow/include\unsupported/Eigen/CXX11/Tensor(74): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory

Describe the expected behavior

Compiling succeed.

My Research

The error occurred in an Eigen C++ file named Tensor:

#ifdef EIGEN_USE_GPU
  #include <iostream>
  #if defined(EIGEN_USE_HIP)
    #include <hip/hip_runtime.h>
  #else
    #include <cuda_runtime.h>
  #endif
  #include <atomic>
  #include <unistd.h>
#endif

You can see that unistd.h including is not excluded by some conditional compiling branch of _WIN32.

It seems a bug within Eigen, but I found the original source here:

#ifdef EIGEN_USE_GPU
#include <iostream>
#include <cuda_runtime.h>
#if __cplusplus >= 201103L
#include <atomic>
#include <unistd.h>
#endif
#endif

This is different, and I haven't found any uniform record in their committing history. So this seems modified by Tensorflow team or someone else already.

And I solved this issue just by commenting out the line of #include <unistd.h> in the tensorflow python installation directory. Expecting the official fix.

[ravikyram]

@k-l-lambda

Can you please refer link1 and link2 and see if it helps you.Thanks!

[k-l-lambda]

@k-l-lambda

Can you please refer link1 and link2 and see if it helps you.Thanks!

No, it doesn't help.

The simple workaround is just remove the include line of unistd.h. But when I publish a middle-ware library, client developers expect that can work directly, without some bothering manual edition in a third-party source code, following a complicated instruction list.

Some one should take the responsibility to correct this, if not tensorflow team, please tell me the where the code come from.

C:/Users/xxx/AppData/Local/Programs/Python/Python36/lib/site-packages/tensorflow/include\unsupported/Eigen/CXX11/Tensor

Please read my issue description for details.

[sushreebarsa]

@k-l-lambda It looks like you are using an older Version of Tensorflow (2.2). Many bugs have been fixed in the latest version. Could you please execute your code using Latest Version (2.6) and let us know if the issue still persists? Thanks!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No