Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tf2 isn't enabled in tensorflow_core.python.keras.layers.__init__ #36700

Closed
EmGarr opened this issue Feb 12, 2020 · 8 comments
Closed

tf2 isn't enabled in tensorflow_core.python.keras.layers.__init__ #36700

EmGarr opened this issue Feb 12, 2020 · 8 comments
Assignees
@tanzhenyu
Labels
comp:keras Keras related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.1 for tracking issues in 2.1 release type:bug Bug
Projects
TensorFlow 2.2.0

Comments

@EmGarr
Copy link

EmGarr commented Feb 12, 2020 •
edited

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
  • TensorFlow installed from (source or binary): pip install tensorflow==2.1
  • TensorFlow version (use command below): 2.1
  • Python version: 3.6.6
  • CUDA/cuDNN version: Not used
  • GPU model and memory: Not used

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior

Whenever you import a layer using the path:
from tensorflow.python.keras.layers

It will import the layer using the tensorflow 1.X behavior.
Which isn't the case when we use tensorflow.keras.layers.

The issue is that every networks from tf.keras.applications (resnet, densenet...) use those import which can lead to some severe bugs (e.g: BatchNormalization).

Describe the expected behavior
Importing from from tensorflow.python.keras.layers and from tensorflow.keras.layers should have exactly the same behavior (2.X).

Code to reproduce the issue

from tensorflow.python.keras.layers import BatchNormalization as buggy_BN
from tensorflow.keras.layers import BatchNormalization as good_BN

print(good_BN()._USE_V2_BEHAVIOR) # TRUE
print(buggy_BN()._USE_V2_BEHAVIOR) # FALSE

Other info / logs

I think that the issue could be fix by changing this tensorflow_core.python.tf2.

from

def enabled():
  """Returns True iff TensorFlow 2.0 behavior should be enabled."""
  if _force_enable is None:
    return os.getenv("TF2_BEHAVIOR", "0") != "0"
  else:
    return _force_enable

to

def enabled():
  """Returns True iff TensorFlow 2.0 behavior should be enabled."""
  if _force_enable is None:
    return os.getenv("TF2_BEHAVIOR", "1") != "0"
  else:
    return _force_enable

it will make TF2_BEHAVIOR enabled by default
@EmGarr EmGarr changed the title tf2 isn't enabled in tensorflow._core.python.keras.layers.__init__ tf2 isn't enabled in tensorflow_core.python.keras.layers.__init__ Feb 12, 2020
@amahendrakar
Copy link
Contributor

Was able to reproduce the issue. Please find the Gist here. Thanks!

@amahendrakar amahendrakar added comp:keras Keras related issues TF 2.1 for tracking issues in 2.1 release type:bug Bug labels Feb 13, 2020
@jvishnuvardhan jvishnuvardhan added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Feb 13, 2020
@jvishnuvardhan
Copy link
Contributor

@EmGarr Importing using tensorflow_core. is not a suggested way for importing any module as mentioned here and another resource. Thanks!

@EmGarr
Copy link
Author

EmGarr commented Feb 13, 2020 •
edited

@jvishnuvardhan I do not use this layer but however all the base model of tensorflow keras do use those imports. It means that whenever you use a Resnet50 (or others) you'll use the batch normalization from tf 1.X which has a totally different behavior for trainable is False.

@artokarj
Copy link

@jvishnuvardhan I do not use this layer but however all the base model of tensorflow keras do use those imports. It means that whenever you use a Resnet50 (or others) you'll use the batch normalization from tf 1.X which has a totally different behavior for trainable is False.

I encountered this problem just yesterday when attempting to use transfer learning with pretrained models from tensorflow.keras.application-package. The validation loss and accuracy aren’t progressing as one would expect (except with VGG16 and others which don’t use BN). Still after loading the model weights I manually set this flag to be true for each BN layers and after that everything seemed to work as one would expect.

This was referenced Mar 17, 2020
Closed
Closed
@raghavab1992
Copy link

@EmGarr we have encountered this issue and was able to solve this as mentioned here

However this has to be fixed by tf team for good as all pretrtained keras models are referencing v1 layers and giving bad results

@goldiegadde goldiegadde added this to In progress in TensorFlow 2.2.0 Mar 25, 2020
TensorFlow 2.2.0 automation moved this from In progress to Done Mar 25, 2020
@tensorflow-bot
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

@goldiegadde goldiegadde mentioned this issue Mar 26, 2020
Closed
@goldiegadde goldiegadde reopened this Mar 26, 2020
TensorFlow 2.2.0 automation moved this from Done to In progress Mar 26, 2020
@goldiegadde
Copy link
Contributor

this issue was fixed in this commit

and the fix has been cherrypicked into r2.2 branch.

@tensorflow-bot
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

TensorFlow 2.2.0 automation moved this from In progress to Done Mar 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tanzhenyu tanzhenyu

Labels
comp:keras Keras related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.1 for tracking issues in 2.1 release type:bug Bug
Projects
TensorFlow 2.2.0
  
Done
Milestone
No milestone
Development

No branches or pull requests
7 participants
@EmGarr @tanzhenyu @raghavab1992 @goldiegadde @jvishnuvardhan @artokarj @amahendrakar