Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pip install runtime error: Couldn't compile and install ats.ops.extract_patches.libpatches #3

Open
Jaiczay opened this issue Aug 13, 2019 · 4 comments
Open

pip install runtime error: Couldn't compile and install ats.ops.extract_patches.libpatches #3

Jaiczay opened this issue Aug 13, 2019 · 4 comments

Comments

@Jaiczay
Copy link

Jaiczay commented Aug 13, 2019

On Ubuntu 19.04
Python 3.7.3

pip3 install attention-sampling
Collecting attention-sampling
  Using cached https://files.pythonhosted.org/packages/ff/2d/4474d1f516865eb83419c67045d885adc03266f97858fa8eeb14117c709d/attention-sampling-0.2.tar.gz
Requirement already satisfied: keras>=2 in /home/jaiczay/.local/lib/python3.7/site-packages (from attention-sampling) (2.2.4)
Requirement already satisfied: numpy in /home/jaiczay/.local/lib/python3.7/site-packages (from attention-sampling) (1.16.4)
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from keras>=2->attention-sampling) (3.13)
Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from keras>=2->attention-sampling) (1.12.0)
Requirement already satisfied: h5py in /home/jaiczay/.local/lib/python3.7/site-packages (from keras>=2->attention-sampling) (2.9.0)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /home/jaiczay/.local/lib/python3.7/site-packages (from keras>=2->attention-sampling) (1.1.0)
Requirement already satisfied: scipy>=0.14 in /home/jaiczay/.local/lib/python3.7/site-packages (from keras>=2->attention-sampling) (1.3.1)
Requirement already satisfied: keras-applications>=1.0.6 in /home/jaiczay/.local/lib/python3.7/site-packages (from keras>=2->attention-sampling) (1.0.7)
Building wheels for collected packages: attention-sampling
  Running setup.py bdist_wheel for attention-sampling ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-pbfqhw21/attention-sampling/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-axu0ud85 --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/ats
  copying ats/__init__.py -> build/lib.linux-x86_64-3.7/ats
  creating build/lib.linux-x86_64-3.7/ats/ops
  copying ats/ops/__init__.py -> build/lib.linux-x86_64-3.7/ats/ops
  creating build/lib.linux-x86_64-3.7/ats/utils
  copying ats/utils/layers.py -> build/lib.linux-x86_64-3.7/ats/utils
  copying ats/utils/regularizers.py -> build/lib.linux-x86_64-3.7/ats/utils
  copying ats/utils/training.py -> build/lib.linux-x86_64-3.7/ats/utils
  copying ats/utils/__init__.py -> build/lib.linux-x86_64-3.7/ats/utils
  creating build/lib.linux-x86_64-3.7/ats/core
  copying ats/core/sampling.py -> build/lib.linux-x86_64-3.7/ats/core
  copying ats/core/builder.py -> build/lib.linux-x86_64-3.7/ats/core
  copying ats/core/expectation.py -> build/lib.linux-x86_64-3.7/ats/core
  copying ats/core/ats_layer.py -> build/lib.linux-x86_64-3.7/ats/core
  copying ats/core/__init__.py -> build/lib.linux-x86_64-3.7/ats/core
  creating build/lib.linux-x86_64-3.7/ats/data
  copying ats/data/from_tensors.py -> build/lib.linux-x86_64-3.7/ats/data
  copying ats/data/base.py -> build/lib.linux-x86_64-3.7/ats/data
  copying ats/data/__init__.py -> build/lib.linux-x86_64-3.7/ats/data
  creating build/lib.linux-x86_64-3.7/ats/ops/extract_patches
  copying ats/ops/extract_patches/__init__.py -> build/lib.linux-x86_64-3.7/ats/ops/extract_patches
  running build_ext
  Building ats.ops.extract_patches.libpatches
  -- The C compiler identification is GNU 8.3.0
  -- The CXX compiler identification is GNU 8.3.0
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  CUDA_TOOLKIT_ROOT_DIR not found or specified
  -- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
    File "<string>", line 1
      import tensorflow as tf;         print(tf.sysconfig.get_include(), end='')
                                                                            ^
  SyntaxError: invalid syntax
  -- Found TensorFlow include:
    File "<string>", line 1
      import tensorflow as tf;         print(tf.sysconfig.get_lib(), end='')
                                                                        ^
  SyntaxError: invalid syntax
    File "<string>", line 1
      import tensorflow as tf;         lib = next(f for f in tf.sysconfig.get_link_flags()                    if f.startswith('-l'));         print(lib[2:] if lib[2] != ':' else lib[3:], end='')
                                                                                                                                                                                              ^
  SyntaxError: invalid syntax
  -- Found TensorFlow lib: TENSORFLOW_LIB-NOTFOUND
    File "<string>", line 1
      import tensorflow as tf;                 print(tf.__version__, end='')
                                                                        ^
  SyntaxError: invalid syntax
  CMake Warning at /tmp/pip-install-pbfqhw21/attention-sampling/ats/ops/cmake/FindTensorFlow.cmake:60 (message):
    You are using an unsupported compiler! It is recommended to build your
    TensorFlow extensions with gcc 4.8 .
  Call Stack (most recent call first):
    CMakeLists.txt:14 (find_package)
  
  
    File "<string>", line 1
      import tensorflow as tf;         print(' '.join(             f for f in tf.sysconfig.get_compile_flags()             if not f.startswith('-I')), end='')
                                                                                                                                                          ^
  SyntaxError: invalid syntax
  -- Added TensorFlow flags:
  CMake Error at /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
    Could NOT find TensorFlow (missing: TENSORFLOW_INCLUDE_DIR TENSORFLOW_LIB)
  Call Stack (most recent call first):
    /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
    /tmp/pip-install-pbfqhw21/attention-sampling/ats/ops/cmake/FindTensorFlow.cmake:82 (find_package_handle_standard_args)
    CMakeLists.txt:14 (find_package)
  
  
  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-pbfqhw21/attention-sampling/ats/ops/extract_patches/build/CMakeFiles/CMakeOutput.log".
  Traceback (most recent call last):
    File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 33, in compile
      cwd=extension_dir
    File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '-DCMAKE_BUILD_TYPE=Release', '..']' returned non-zero exit status 1.
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 144, in <module>
      setup_package()
    File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 139, in setup_package
      cmdclass={"build_ext": custom_build_ext}
    File "/home/jaiczay/.local/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 188, in run
      self.run_command('build')
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/home/jaiczay/.local/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 78, in run
      _build_ext.run(self)
    File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 52, in build_extension
      ext.compile()
    File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 46, in compile
      ) from e
  RuntimeError: Couldn't compile and install ats.ops.extract_patches.libpatches
  
  ----------------------------------------
  Failed building wheel for attention-sampling
  Running setup.py clean for attention-sampling
Failed to build attention-sampling
Installing collected packages: attention-sampling
  Running setup.py install for attention-sampling ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-pbfqhw21/attention-sampling/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-n262zdr0/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/ats
    copying ats/__init__.py -> build/lib.linux-x86_64-3.7/ats
    creating build/lib.linux-x86_64-3.7/ats/ops
    copying ats/ops/__init__.py -> build/lib.linux-x86_64-3.7/ats/ops
    creating build/lib.linux-x86_64-3.7/ats/utils
    copying ats/utils/layers.py -> build/lib.linux-x86_64-3.7/ats/utils
    copying ats/utils/regularizers.py -> build/lib.linux-x86_64-3.7/ats/utils
    copying ats/utils/training.py -> build/lib.linux-x86_64-3.7/ats/utils
    copying ats/utils/__init__.py -> build/lib.linux-x86_64-3.7/ats/utils
    creating build/lib.linux-x86_64-3.7/ats/core
    copying ats/core/sampling.py -> build/lib.linux-x86_64-3.7/ats/core
    copying ats/core/builder.py -> build/lib.linux-x86_64-3.7/ats/core
    copying ats/core/expectation.py -> build/lib.linux-x86_64-3.7/ats/core
    copying ats/core/ats_layer.py -> build/lib.linux-x86_64-3.7/ats/core
    copying ats/core/__init__.py -> build/lib.linux-x86_64-3.7/ats/core
    creating build/lib.linux-x86_64-3.7/ats/data
    copying ats/data/from_tensors.py -> build/lib.linux-x86_64-3.7/ats/data
    copying ats/data/base.py -> build/lib.linux-x86_64-3.7/ats/data
    copying ats/data/__init__.py -> build/lib.linux-x86_64-3.7/ats/data
    creating build/lib.linux-x86_64-3.7/ats/ops/extract_patches
    copying ats/ops/extract_patches/__init__.py -> build/lib.linux-x86_64-3.7/ats/ops/extract_patches
    running build_ext
    Building ats.ops.extract_patches.libpatches
    CUDA_TOOLKIT_ROOT_DIR not found or specified
    -- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
      File "<string>", line 1
        import tensorflow as tf;         print(tf.sysconfig.get_include(), end='')
                                                                              ^
    SyntaxError: invalid syntax
    -- Found TensorFlow include:
      File "<string>", line 1
        import tensorflow as tf;         print(tf.sysconfig.get_lib(), end='')
                                                                          ^
    SyntaxError: invalid syntax
      File "<string>", line 1
        import tensorflow as tf;         lib = next(f for f in tf.sysconfig.get_link_flags()                    if f.startswith('-l'));         print(lib[2:] if lib[2] != ':' else lib[3:], end='')
                                                                                                                                                                                                ^
    SyntaxError: invalid syntax
    -- Found TensorFlow lib: TENSORFLOW_LIB-NOTFOUND
      File "<string>", line 1
        import tensorflow as tf;                 print(tf.__version__, end='')
                                                                          ^
    SyntaxError: invalid syntax
    CMake Warning at /tmp/pip-install-pbfqhw21/attention-sampling/ats/ops/cmake/FindTensorFlow.cmake:60 (message):
      You are using an unsupported compiler! It is recommended to build your
      TensorFlow extensions with gcc 4.8 .
    Call Stack (most recent call first):
      CMakeLists.txt:14 (find_package)
    
    
      File "<string>", line 1
        import tensorflow as tf;         print(' '.join(             f for f in tf.sysconfig.get_compile_flags()             if not f.startswith('-I')), end='')
                                                                                                                                                            ^
    SyntaxError: invalid syntax
    -- Added TensorFlow flags:
    CMake Error at /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
      Could NOT find TensorFlow (missing: TENSORFLOW_INCLUDE_DIR TENSORFLOW_LIB)
    Call Stack (most recent call first):
      /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
      /tmp/pip-install-pbfqhw21/attention-sampling/ats/ops/cmake/FindTensorFlow.cmake:82 (find_package_handle_standard_args)
      CMakeLists.txt:14 (find_package)
    
    
    -- Configuring incomplete, errors occurred!
    See also "/tmp/pip-install-pbfqhw21/attention-sampling/ats/ops/extract_patches/build/CMakeFiles/CMakeOutput.log".
    Traceback (most recent call last):
      File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 33, in compile
        cwd=extension_dir
      File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '-DCMAKE_BUILD_TYPE=Release', '..']' returned non-zero exit status 1.
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 144, in <module>
        setup_package()
      File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 139, in setup_package
        cmdclass={"build_ext": custom_build_ext}
      File "/home/jaiczay/.local/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/home/jaiczay/.local/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.7/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/home/jaiczay/.local/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 78, in run
        _build_ext.run(self)
      File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 52, in build_extension
        ext.compile()
      File "/tmp/pip-install-pbfqhw21/attention-sampling/setup.py", line 46, in compile
        ) from e
    RuntimeError: Couldn't compile and install ats.ops.extract_patches.libpatches
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-pbfqhw21/attention-sampling/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-n262zdr0/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-pbfqhw21/attention-sampling/
@angeloskath
Copy link
Collaborator

Hi,

Thanks a lot for the issue! Could you run the following to double-check what the error is?

python -c "import tensorflow as tf; print(tf.sysconfig.get_include(), end='')"

I will try to replicate the issue and have it fixed.

Cheers,
Angelos

@Jaiczay
Copy link
Author

Jaiczay commented Aug 14, 2019

Thx for the reply.
This is the output:

/home/jaiczay/.local/lib/python3.7/site-packages/tensorflow/include

@angeloskath
Copy link
Collaborator

Hi again,

I am struggling to reproduce the error, however because several people are having trouble compiling the tf extension. I will try to create precompiled binary distributions for both anaconda and PyPI by the end of the month.

For now, I would suggest

  • using tensorflow 1.13.1
  • making sure that you have
    • CMake installed
    • g++ installed
    • tensorflow installed

Let me know if you need anything more.

Cheers,
Angelos

@lone17
Copy link

lone17 commented Oct 31, 2019

I had the same problem and it f*ck me up for 2 days but i finally got a successful installation. The cause of this problem may vary and it looks like mine is different from yours, but here are some suggestions:

  • i'm using tf 1.13.1 and python 3.6.8 and cuda 10.0. based on my experience, i recommend you stick with python 3.6 instead of 3.7 when working with tf and cuda
  • make sure you can use tensorflow, i.e. be able to import tensorflow
  • if you are using tensorflow-gpu, make sure the cuda versions form nvcc --version and nvidia-smi are the same
  • try cloning the repo and build it manually, i.e. run python setup.py install. you might also want to add this line set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11") to the start of attention-sampling/ats/ops/extract_patches/CMakeLists.txt, i add it at line 6.

best of luck !

@lone17 lone17 mentioned this issue Oct 31, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@angeloskath @lone17 @Jaiczay