ModuleNotFoundError: No module named 'tensorflow.python.keras.applications'

[mmonto95eafit]

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/model_main_tf2.py

2. Describe the bug

After the latest release there is a ModuleNotFoundError after following the Python Package Installation Instructions

3. Steps to reproduce

1. Install object detection API as per the Python Package Installation Instructions:

git clone https://github.com/tensorflow/models.git

cd models/research
# Compile protos.
protoc object_detection/protos/*.proto --python_out=.
# Install TensorFlow Object Detection API.
cp object_detection/packages/tf2/setup.py .
python -m pip install --use-feature=2020-resolver .

2. Test the installation with the test code.

python object_detection/builders/model_builder_tf2_test.py

3. The error shows up:

Traceback (most recent call last):
  File "object_detection/builders/model_builder_tf2_test.py", line 25, in <module>
    from object_detection.builders import model_builder
  File "/usr/local/lib/python3.7/dist-packages/object_detection/builders/model_builder.py", line 37, in <module>
    from object_detection.meta_architectures import deepmac_meta_arch
  File "/usr/local/lib/python3.7/dist-packages/object_detection/meta_architectures/deepmac_meta_arch.py", line 19, in <module>
    from object_detection.models.keras_models import resnet_v1
  File "/usr/local/lib/python3.7/dist-packages/object_detection/models/keras_models/resnet_v1.py", line 22, in <module>
    from tensorflow.python.keras.applications import resnet
ModuleNotFoundError: No module named 'tensorflow.python.keras.applications'

4. Temporal workaround

After cloning the repository, go to object_detection/models/keras_models/resnet_v1.py and comment line 22. Then proceed building as usual with the setup.py (Note that this may cause many errors that I am not taking into account, but it served for my purpose of traning an efficientDet model).

5. System information

• This happened to me using google colab and using some other machines I rented. I believe it's happening anywhere.

[mmonto95eafit]

This is related to #10479

[nguaki]

I got that error also.
Training on SSD_MobileNet.

But when I run it for an image detection, i got this error message:
detections = detect_fn(input_tensor)
23
24 num_detections = int(detections.pop('num_detections'))

4 frames
/usr/local/lib/python3.7/dist-packages/object_detection/models/ssd_mobilenet_v2_fpn_keras_feature_extractor.py in _extract_features(self, preprocessed_inputs)
218
219 image_features = self.classification_backbone(
--> 220 ops.pad_to_multiple(preprocessed_inputs, self._pad_to_multiple))
221
222 feature_block_list = []

UnimplementedError: Exception encountered when calling layer "Conv1" (type Conv2D).

Is this error due to new release of tensorflow?
Why Colab does not make an adjustment and have the users suffer?

[SauBuen]

After doing This is related to #10479
and trying:

REQUIRED_PACKAGES = [
# Required for apache-beam with PY3
'avro-python3',
'apache-beam',
'pillow',
'lxml',
'matplotlib',
'Cython',
'contextlib2',
'tf-slim==1.1.0',
'six',
'pycocotools',
'lvis',
'scipy',
'pandas',
'tf-models-official==2.7.0',
'tensorflow_io==0.23.1',
'keras==2.7.0'
]

2022-02-16 06:37:40.676005: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
I0216 06:37:40.679783 139960035796864 mirrored_strategy.py:376] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
INFO:tensorflow:Maybe overwriting train_steps: 40000
I0216 06:37:40.687003 139960035796864 config_util.py:552] Maybe overwriting train_steps: 40000
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0216 06:37:40.687190 139960035796864 config_util.py:552] Maybe overwriting use_bfloat16: False
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py:564: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
W0216 06:37:40.717741 139960035796864 deprecation.py:347] From /usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py:564: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
INFO:tensorflow:Reading unweighted datasets: ['Tensorflow/workspace/annotations/train_v2.tfrecord']
I0216 06:37:40.728104 139960035796864 dataset_builder.py:163] Reading unweighted datasets: ['Tensorflow/workspace/annotations/train_v2.tfrecord']
INFO:tensorflow:Reading record datasets for input file: ['Tensorflow/workspace/annotations/train_v2.tfrecord']
I0216 06:37:40.728515 139960035796864 dataset_builder.py:80] Reading record datasets for input file: ['Tensorflow/workspace/annotations/train_v2.tfrecord']
INFO:tensorflow:Number of filenames to read: 1
I0216 06:37:40.728646 139960035796864 dataset_builder.py:81] Number of filenames to read: 1
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W0216 06:37:40.728734 139960035796864 dataset_builder.py:88] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.deterministic`.
W0216 06:37:40.731439 139960035796864 deprecation.py:347] From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.deterministic`.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
W0216 06:37:40.752431 139960035796864 deprecation.py:347] From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:1096: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
W0216 06:37:48.094985 139960035796864 deprecation.py:347] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:1096: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:1096: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
W0216 06:37:51.339461 139960035796864 deprecation.py:347] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:1096: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py:465: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0216 06:37:53.080455 139960035796864 deprecation.py:347] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py:465: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
/usr/local/lib/python3.7/dist-packages/keras/backend.py:414: UserWarning: `tf.keras.backend.set_learning_phase` is deprecated and will be removed after 2020-10-11. To update it, simply pass a True/False value to the `training` argument of the `__call__` method of your layer or model.
  warnings.warn('`tf.keras.backend.set_learning_phase` is deprecated and '
2022-02-16 06:39:02.982969: E tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0.  CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
2022-02-16 06:39:02.984087: W tensorflow/core/framework/op_kernel.cc:1745] OP_REQUIRES failed at conv_ops.cc:1120 : UNKNOWN: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
Traceback (most recent call last):
  File "Tensorflow/models/research/object_detection/model_main_tf2.py", line 115, in <module>
    tf.compat.v1.app.run()
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "Tensorflow/models/research/object_detection/model_main_tf2.py", line 112, in main
    record_summaries=FLAGS.record_summaries)
  File "/usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py", line 609, in train_loop
    train_input, unpad_groundtruth_tensors)
  File "/usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py", line 400, in load_fine_tune_checkpoint
    _ensure_model_is_built(model, input_dataset, unpad_groundtruth_tensors)
  File "/usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py", line 178, in _ensure_model_is_built
    labels,
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py", line 1316, in run
    return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py", line 2892, in call_for_each_replica
    return self._call_for_each_replica(fn, args, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/mirrored_strategy.py", line 678, in _call_for_each_replica
    self._container_strategy(), fn, args, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/mirrored_run.py", line 86, in call_for_each_replica
    return wrapped(args, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.UnknownError:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
	 [[node ssd_mobile_net_v2_fpn_keras_feature_extractor/model/Conv1/Conv2D
 (defined at /usr/local/lib/python3.7/dist-packages/keras/layers/convolutional.py:238)
]] [Op:__inference__dummy_computation_fn_44614]

Errors may have originated from an input operation.
Input Source operations connected to node ssd_mobile_net_v2_fpn_keras_feature_extractor/model/Conv1/Conv2D:
In[0] args_1 (defined at /usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py:178)	
In[1] ssd_mobile_net_v2_fpn_keras_feature_extractor/model/Conv1/Conv2D/ReadVariableOp:

Operation defined at: (most recent call last)
>>>   File "/usr/lib/python3.7/threading.py", line 890, in _bootstrap
>>>     self._bootstrap_inner()
>>> 
>>>   File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
>>>     self.run()
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py", line 170, in _dummy_computation_fn
>>>     return _compute_losses_and_predictions_dicts(model, features, labels,
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py", line 123, in _compute_losses_and_predictions_dicts
>>>     prediction_dict = model.predict(
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/object_detection/meta_architectures/ssd_meta_arch.py", line 569, in predict
>>>     if self._feature_extractor.is_keras_model:
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/object_detection/meta_architectures/ssd_meta_arch.py", line 570, in predict
>>>     feature_maps = self._feature_extractor(preprocessed_inputs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
>>>     return fn(*args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1083, in __call__
>>>     outputs = call_fn(inputs, *args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
>>>     return fn(*args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/object_detection/meta_architectures/ssd_meta_arch.py", line 251, in call
>>>     return self._extract_features(inputs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/object_detection/models/ssd_mobilenet_v2_fpn_keras_feature_extractor.py", line 219, in _extract_features
>>>     image_features = self.classification_backbone(
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
>>>     return fn(*args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1083, in __call__
>>>     outputs = call_fn(inputs, *args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
>>>     return fn(*args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 452, in call
>>>     inputs, training=training, mask=mask)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/engine/functional.py", line 589, in _run_internal_graph
>>>     outputs = node.layer(*args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
>>>     return fn(*args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 1083, in __call__
>>>     outputs = call_fn(inputs, *args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
>>>     return fn(*args, **kwargs)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/layers/convolutional.py", line 246, in call
>>>     outputs = self.convolution_op(inputs, self.kernel)
>>> 
>>>   File "/usr/local/lib/python3.7/dist-packages/keras/layers/convolutional.py", line 238, in convolution_op
>>>     name=self.__class__.__name__)
>>> 

[saberkun]

@qlzh727 Keras team may help take a look.

[qlzh727]

tensorflow.python.keras package is deprecated and being removed. The keras project code is moved to keras-team/keras. You should only access the public keras API via tf.keras.applications.*.

[nguaki]

@qlzh727 Would you provide an example? Confused which commands are not allowed and which commands are allowed.

[qlzh727]

Allowed:

from tensorflow import keras
keras.applications.xxx


import tensorflow as tf
tf.keras.applications.xxx

Not Allowed:

from tensorflow.python import keras
keras.applications.xxx

from tensorflow.python.keras import applications
applications.xxx

The only difference here is accessing tensorflow.python.keras package directly.

[nguaki]

@qlzh727 thank you very much. this really clarified what to use and what not to use.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No