Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tensorflow eager mode dose not work when GPU is enabled in google colab #262

Closed
alinamadchian opened this issue Sep 9, 2018 · 9 comments

Comments

@alinamadchian
Copy link

recently I have faced this problem.
when I want to use GPU as the accelerator in google collab, the eager mode won't work and when I try to enable the eager mode even at the beginning of the program it gives me the error:
ValueError: tf.enable_eager_execution must be called at program startup.
while when the GPU is not used the eager mode can be activated and be used successfully in google collab

@colaboratory-team
Copy link
Contributor

Sorry for the trouble. Please try the following and let us know if the issue recurs:

  1. Reset your environment using the 'Reset all runtimes' item from the Runtime menu.
    reset

  2. Execute the following in a code cell:

import tensorflow as tf
tf.enable_eager_execution()

@alinamadchian
Copy link
Author

Unfortunately, I tried again and the issue is not resolved. I did not have such a problem two days ago, it seems something goes wrong recently

@craigcitro
Copy link
Contributor

Yep, this is a bug (which will only happen with GPUs) -- to work around it for now, reset all runtimes, and then run

import os
os.environ['DISABLE_COLAB_TF_IMPORT_HOOK'] = '1'

before any TF-using code, and you should be unblocked.

@alinamadchian
Copy link
Author

Yes, It works.
thank you very much.

@craigcitro
Copy link
Contributor

FTR fix is now live, this should be working without the custom env var (after possibly restarting any runtimes).

@ParasharaRamesh
Copy link

I tried the @craigcitro 's solution of importing the os module. But it still gives me an error saying module 'tensorflow' has no attribute 'enable_eager_execution'.
The colab tutorial I am trying is the following
"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/r2/tutorials/generative/image_captioning.ipynb#scrollTo=Dx_fvbVgRPGQ"
and in the cell "Caching the features extracted from InceptionV3" the last line with "np.save(path_of_feature, bf.numpy())" gives me an error saying "Tensor has no attribute called numpy()" , so i am guessing it is to do with enabling eager execution but i am not sure.

@craigcitro
Copy link
Contributor

@ParasharaRamesh Can you file a separate issue for what you're hitting?

@MarkDaoust
Copy link

@ParasharaRamesh Can you file a separate issue for what you're hitting?

Yes please.

Everything in that r2/ directory is updates in progress for tf2.0.
Not everything works yet. But this looks worse than an error in the notebook.

That batch_features = image_features_extract_model(img) line is returning a graph tensor, instead of an eager tensor. It's not supposed to do that.

Can you raise an issue on tensorflow/tensorflow, and cc me on it?

@ParasharaRamesh
Copy link

@MarkDaoust I was able to get past that issue by doing the following:

  • Not pip installing tf-nightly
  • using the default tensorflow 1.13.0rc version and then executing tf.enable_eager_execution() which happened to work where i was previously getting an error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants