New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.10 build fails with "No module named 'keras_applications'" #21518

Closed
skeydan opened this Issue Aug 9, 2018 · 27 comments

Comments

Projects
None yet
9 participants
@skeydan

skeydan commented Aug 9, 2018

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Fedora 28
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
  • TensorFlow installed from (source or binary): source
  • TensorFlow version (use command below): 1.10
  • Python version: 3.6
  • Bazel version (if compiling from source): 0.16
  • GCC/Compiler version (if compiling from source): 7.3
  • CUDA/cuDNN version: 9.2
  • GPU model and memory: Quadro M2200 4G
  • Exact command to reproduce: bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

Hi, a recent checkout of master (~ 2 hours ago) fails (for me) with

ERROR: /home/key/code/tensorflow/tensorflow/BUILD:584:1: Executing genrule //tensorflow:tensorflow_python_api_gen failed (Exit 1)
/home/key/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Traceback (most recent call last):
  File "/home/key/.cache/bazel/_bazel_key/8de93c72df6c3fec3e289f10fadff72b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/tools/api/generator/create_python_api.py", line 27, in <module>
    from tensorflow.python.tools.api.generator import doc_srcs
  File "/home/key/.cache/bazel/_bazel_key/8de93c72df6c3fec3e289f10fadff72b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 81, in <module>
    from tensorflow.python import keras
  File "/home/key/.cache/bazel/_bazel_key/8de93c72df6c3fec3e289f10fadff72b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/__init__.py", line 25, in <module>
    from tensorflow.python.keras import applications
  File "/home/key/.cache/bazel/_bazel_key/8de93c72df6c3fec3e289f10fadff72b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/applications/__init__.py", line 21, in <module>
    import keras_applications
ModuleNotFoundError: No module named 'keras_applications'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 3855.754s, Critical Path: 156.93s
INFO: 8932 processes: 8932 local.
FAILED: Build did NOT complete successfully

Could you please advise?

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 9, 2018

I believe the following will fix this issue:

pip install keras_applications==1.0.4 --no-deps
pip install keras_preprocessing==1.0.2 --no-deps

@yifeif any additional info?

@av8ramit av8ramit self-assigned this Aug 9, 2018

@yifeif

This comment has been minimized.

Member

yifeif commented Aug 9, 2018

You might also need pip install h5py==2.8.0

@gunan

This comment has been minimized.

Member

gunan commented Aug 9, 2018

we should update our installation from sources documentation with this information.

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 9, 2018

#21527

@skeydan please let us know if this helps.

@geekonedge

This comment has been minimized.

geekonedge commented Aug 9, 2018

I can confirm that installing the three packages resolves the issue as per this link

@skeydan

This comment has been minimized.

skeydan commented Aug 10, 2018

works great, thank you!!

@av8ramit av8ramit closed this Aug 10, 2018

@claynerobison

This comment has been minimized.

Contributor

claynerobison commented Aug 10, 2018

@gunan Are you going to update the Dockerfiles or do you want me to submit a PR for it? All the containerized builds are failing too.

@gunan

This comment has been minimized.

Member

gunan commented Aug 10, 2018

AH, I completely missed that.
@av8ramit, could you send a PR to update all our dockerfiles to install the 3 pip packages mentioned in this issue?

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 10, 2018

Sure, sounds good.

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 14, 2018

Sent internally.

@av8ramit av8ramit closed this Aug 14, 2018

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 14, 2018

This issue has now been fixed.

@owengray-brainco

This comment has been minimized.

owengray-brainco commented Aug 16, 2018

I am still having this issue. I have installed keras(-applications) and h5py according to this.
Running a fresh pull of master with most recent commit 19cafed 62191da Thu Aug 16 13:04:52
Running
bazel build -c opt --copt=-march=native //tensorflow/tools/pip_package:build_pip_package
The error ends with
ERROR: /Users/owengray/Desktop/git/tensorflow/tensorflow/BUILD:576:1: Executing genrule //tensorflow:tensorflow_python_api_gen failed (Exit 1) Traceback (most recent call last): File "/private/var/tmp/_bazel_owengray/8e2e063d880456f5e168abd014557d0b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/tools/api/generator/create_python_api.py", line 27, in <module> from tensorflow.python.tools.api.generator import doc_srcs File "/private/var/tmp/_bazel_owengray/8e2e063d880456f5e168abd014557d0b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 81, in <module> from tensorflow.python import keras File "/private/var/tmp/_bazel_owengray/8e2e063d880456f5e168abd014557d0b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/__init__.py", line 25, in <module> from tensorflow.python.keras import applications File "/private/var/tmp/_bazel_owengray/8e2e063d880456f5e168abd014557d0b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/applications/__init__.py", line 21, in <module> import keras_applications ModuleNotFoundError: No module named 'keras_applications' Target //tensorflow/tools/pip_package:build_pip_package failed to build

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 16, 2018

What python version are you running with? What was the output of your pip install commands?

@owengray-brainco

This comment has been minimized.

owengray-brainco commented Aug 16, 2018

Owens-MBP:~ owengray$ pip install keras_applications==1.0.4 --no-deps Requirement already satisfied: keras_applications==1.0.4 in /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (1.0.4) Owens-MBP:~ owengray$ pip install keras_preprocessing==1.0.2 --no-deps Requirement already satisfied: keras_preprocessing==1.0.2 in /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (1.0.2) Owens-MBP:~ owengray$ pip install h5py==2.8.0 Requirement already satisfied: h5py==2.8.0 in /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (2.8.0) Requirement already satisfied: numpy>=1.7 in /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from h5py==2.8.0) (1.15.0) Requirement already satisfied: six in /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from h5py==2.8.0) (1.11.0)
Owens-MBP:~ owengray$ /usr/local/bin/python3 --version Python 3.6.5

@owengray-brainco

This comment has been minimized.

owengray-brainco commented Aug 16, 2018

Also,
Owens-MBP:tensorflow owengray$ ./configure WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown". You have bazel 0.16.1 installed. Please specify the location of python. [Default is /usr/bin/python]: /usr/local/bin/python3 Found possible Python library paths: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages Please input the desired Python library path to use. Default is [/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages]

I get several errors running bazel fetch ... -k, but they don't seem related.

EDIT: Actually, it looks like I may have answered my own question--not reading library paths clearly enough (/Library vs /opt/local/Library)

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 16, 2018

Yup, that looks like it's it. Please let us know if it works now.

@owengray-brainco

This comment has been minimized.

owengray-brainco commented Aug 17, 2018

Still didn't work.
Found possible Python library paths: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages Please input the desired Python library path to use. Default is [/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages] /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages

File "/private/var/tmp/_bazel_owengray/8e2e063d880456f5e168abd014557d0b/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/applications/__init__.py", line 21, in <module> import keras_applications ModuleNotFoundError: No module named 'keras_applications' Target //tensorflow/tools/pip_package:build_pip_package failed to build

@av8ramit

This comment has been minimized.

Member

av8ramit commented Aug 17, 2018

My theory is you're not installing keras_applications to the right python path that you're specifying.

@owengray-brainco

This comment has been minimized.

owengray-brainco commented Aug 17, 2018

Using tab autocomplete:
/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras keras/ keras_applications/ keras_preprocessing/

@jeanpat

This comment has been minimized.

jeanpat commented Aug 23, 2018

What about building tensorflow in a virtual environment?

@gunan

This comment has been minimized.

Member

gunan commented Aug 23, 2018

You will either need to create the virtualenv with --system-site-packages, or run pip install keras-applications (and other pip install commands) after you activate the virtualenv.

@mpekalski

This comment has been minimized.

mpekalski commented Aug 26, 2018

I got the same issue while building the latest master. Why is bazel not taking care of it?

@gunan

This comment has been minimized.

Member

gunan commented Aug 26, 2018

That is an interesting point. I think the answer is we have not looked into making this package available through our bazel workspace.
I would be thrilled to review such a contribution to the repo. Unfortunately, I have a few high priority items on my list that make me unable to work on it.

@mpekalski

This comment has been minimized.

mpekalski commented Sep 5, 2018

I am building TF with debug symbols, the latest greatest commit, and it looks like Tensorflow requires

    'keras_applications >= 1.0.5',
    'keras_preprocessing >= 1.0.3',

but keras itself wants different version

error: Keras-Applications 1.0.5 is installed but keras-applications==1.0.4 is required by {'keras'}

https://github.com/tensorflow/tensorflow/blob/6729cd7d1c07e547298fa2a02d1e36390dc62f0a/tensorflow/tools/pip_package/setup.py

% pip install keras_applications==1.0.4  --no-deps
Requirement already satisfied: keras_applications==1.0.4 in ~/.pyenv/versions/venv36/lib/python3.5/site-packages (1.0.4)
                 
% pip install keras_preprocessing==1.0.2  --no-deps
Requirement already satisfied: keras_preprocessing==1.0.2 in ~/.pyenv/versions/env36/lib/python3.5/site-packages (1.0.2)
                          
% python3 setup.py develop
running develop
running egg_info
writing entry points to tensorflow.egg-info/entry_points.txt
writing requirements to tensorflow.egg-info/requires.txt
writing top-level names to tensorflow.egg-info/top_level.txt
writing tensorflow.egg-info/PKG-INFO
writing dependency_links to tensorflow.egg-info/dependency_links.txt
reading manifest file 'tensorflow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.py' under directory '*'
warning: no files found matching '*.pd' under directory '*'
warning: no files found matching '*.so' under directory '*'
warning: no files found matching '*.dll' under directory '*'
warning: no files found matching '*.lib' under directory '*'
warning: no files found matching '*.csv' under directory '*'
warning: no files found matching '*' under directory 'tensorflow/aux-bin'
warning: no files found matching '*.h' under directory 'tensorflow/include/tensorflow'
warning: no files found matching '*' under directory 'tensorflow/include/Eigen'
warning: no files found matching '*' under directory 'tensorflow/include/external'
warning: no files found matching '*.h' under directory 'tensorflow/include/google'
warning: no files found matching '*' under directory 'tensorflow/include/third_party'
warning: no files found matching '*' under directory 'tensorflow/include/unsupported'
writing manifest file 'tensorflow.egg-info/SOURCES.txt'
running build_ext
Creating ~/.pyenv/versions/venv36/lib/python3.5/site-packages/tensorflow.egg-link (link to .)
tensorflow 1.10.0 is already the active version in easy-install.pth
Installing saved_model_cli script to ~/.pyenv/versions/venv36/bin
Installing tensorboard script to ~/.pyenv/versions/venv36/bin
Installing tflite_convert script to ~/.pyenv/versions/venv36/bin
Installing toco_from_protos script to `/.pyenv/versions/venv36/bin
Installing freeze_graph script to ~/.pyenv/versions/venv36/bin
Installing toco script to ~/.pyenv/versions/venv36/bin

Installed tensorflow/tensorflow/tools/pip_package
Processing dependencies for tensorflow==1.10.0
error: Keras-Applications 1.0.5 is installed but keras-applications==1.0.4 is required by {'keras'}

Even if I downgrade keras_applications and keras_processing TF installation overwrites them and complains that {keras} wants different version.

@owengray-brainco

This comment has been minimized.

owengray-brainco commented Sep 5, 2018

I finally got it working. I just needed to make sure that the version of python I passed to the first question in ./configure could successfully import keras when run from command line (ended up being in Frameworks). I may have also needed to set PYTHONPATH.
I also found that the option --copt=-march=native did not work, and I needed to manually specify e.g. --copt=-mavx2
I also got the keras version mismatch error, but the compilation succeeded and works fine.

@mpekalski

This comment has been minimized.

mpekalski commented Sep 5, 2018

In my case it was the problem that there are two different versions of packages with - and _ after keras so keras_applications and keras-applications. I was looking at one and TF was requiring the other one. The same goes for keras_preprocessing

@av8ramit

This comment has been minimized.

Member

av8ramit commented Sep 10, 2018

Closing this for now. Please comment if this is still an issue for you in virtualenv.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment