Unable to create basic model #8

karllandheer · 2022-01-20T01:05:05Z

Hello, I have the following environments:

tensorflow==1.15.0 and keras==2.3.1 and python==3.6

When I try to run the basic line:

model1 = sm.Unet('resnet34', encoder_weights='imagenet')

I get this error:

File "", line 1, in
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/init.py", line 34, in wrapper
return func(*args, **kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/models/unet.py", line 231, in Unet
**kwargs,
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/backbones/backbones_factory.py", line 102, in get_backbone
model = model_fn(*args, **kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models_factory.py", line 74, in wrapper
return func(*args, **new_kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 321, in ResNet34
**kwargs
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 231, in ResNet
x = layers.BatchNormalization(name='bn_data', **no_scale_bn_params)(img_input)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 824, in call
self._maybe_build(inputs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2146, in _maybe_build
self.build(input_shapes)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/layers/normalization.py", line 289, in build
raise ValueError('Invalid axis: %d' % x)

I have tried many permutations of different versions of keras and tensorflow, but to no avail. Any ideas?

karllandheer · 2022-01-20T01:26:54Z

I should mention that the command sm.set_framework('tf.keras') does not fix this error

ZFTurbo · 2022-01-20T11:38:34Z

I added requirements.txt. And checked on my PC - working ok.

    tensorflow>=2.3.2
    keras_applications>=1.0.8
    classification_models_3D>=1.0.3
    efficientnet_3D>=1.0.2

Please install them and check one more time. Also please uninstall keras:

pip uninstall keras

karllandheer · 2022-01-20T14:10:40Z

Hello, thank you for your response. I have created a fresh environment with tensorflow==2.6.2. I uninstalled keras and have the aforementioned packages with the correct version. I imported tensorflow.keras as keras, and imported segmentation_models_3D, however I now get a different error:

Traceback (most recent call last):
File "", line 1, in
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/segmentation_models_3D/init.py", line 34, in wrapper
return func(*args, **kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/segmentation_models_3D/models/unet.py", line 231, in Unet
**kwargs,
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/segmentation_models_3D/backbones/backbones_factory.py", line 102, in get_backbone
model = model_fn(*args, **kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/classification_models_3D/models_factory.py", line 74, in wrapper
return func(*args, **new_kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 321, in ResNet34
**kwargs
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 231, in ResNet
x = layers.BatchNormalization(name='bn_data', **no_scale_bn_params)(img_input)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 997, in call
input_list)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1135, in _functional_construction_call
inputs, input_masks, args, kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 867, in _keras_tensor_symbolic_call
return self._infer_output_signature(inputs, args, kwargs, input_masks)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 905, in _infer_output_signature
self._maybe_build(inputs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2667, in _maybe_build
self.build(input_shapes) # pylint:disable=not-callable
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D7/lib/python3.6/site-packages/tensorflow/python/keras/layers/normalization/batch_normalization.py", line 316, in build
raise ValueError('Invalid axis: %s' % (self.axis,))
ValueError: Invalid axis: ListWrapper([4])

karllandheer · 2022-01-20T14:17:05Z

I got the same error using tensorflow==1.15.0

ZFTurbo · 2022-01-21T18:18:25Z

Can you run file tst_keras.py from repository? What will be the result?

karllandheer · 2022-01-31T15:21:31Z

Hello, that ran successfully. The missing line that's critical for my environment is:

os.environ["KERAS_BACKEND"] = "tensorflow"

Perhaps that should be mentioned somewhere. Thanks very much for your help

ZFTurbo · 2022-01-31T15:22:55Z

Thanks for info I will add it in README.

nathanielatom · 2022-03-05T09:28:04Z

I added requirements.txt. And checked on my PC - working ok.
    tensorflow>=2.3.2
    keras_applications>=1.0.8
    classification_models_3D>=1.0.3
    efficientnet_3D>=1.0.2
Please install them and check one more time. Also please uninstall keras:
pip uninstall keras

Even after this, on python 3.8 I still get:

ValueError: Invalid axis: 4

from running:

import os
os.environ["KERAS_BACKEND"] = "tensorflow"
import segmentation_models_3D as sm
model1 = sm.Unet('resnet34', encoder_weights='imagenet')

ZFTurbo · 2022-03-05T11:47:10Z

Can you post the full error trace?

nathanielatom · 2022-03-05T19:13:59Z

Sure:

Python 3.8.12 | packaged by conda-forge | (default, Jan 30 2022, 23:53:36) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.1.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import os

In [2]: os.environ["KERAS_BACKEND"] = "tensorflow"

In [3]: import segmentation_models_3D as sm
2022-03-05 14:08:43.699751: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2022-03-05 14:08:43.699913: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Segmentation Models: using `tf.keras` framework.

In [4]: model1 = sm.Unet('resnet34', encoder_weights='imagenet')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [4], in <cell line: 1>()
----> 1 model1 = sm.Unet('resnet34', encoder_weights='imagenet')

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/segmentation_models_3D/__init__.py:34, in inject_global_submodules.<locals>.wrapper(*args, **kwargs)
     32 kwargs['models'] = _KERAS_MODELS
     33 kwargs['utils'] = _KERAS_UTILS
---> 34 return func(*args, **kwargs)

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/segmentation_models_3D/models/unet.py:226, in Unet(backbone_name, input_shape, classes, activation, weights, encoder_weights, encoder_freeze, encoder_features, decoder_block_type, decoder_filters, decoder_use_batchnorm, dropout, **kwargs)
    222 else:
    223     raise ValueError('Decoder block type should be in ("upsampling", "transpose"). '
    224                      'Got: {}'.format(decoder_block_type))
--> 226 backbone = Backbones.get_backbone(
    227     backbone_name,
    228     input_shape=input_shape,
    229     weights=encoder_weights,
    230     include_top=False,
    231     **kwargs,
    232 )
    234 if encoder_features == 'default':
    235     encoder_features = Backbones.get_feature_layers(backbone_name, n=4)

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/segmentation_models_3D/backbones/backbones_factory.py:102, in BackbonesFactory.get_backbone(self, name, *args, **kwargs)
    100 def get_backbone(self, name, *args, **kwargs):
    101     model_fn, _ = self.get(name)
--> 102     model = model_fn(*args, **kwargs)
    103     return model

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/classification_models_3D/models_factory.py:74, in ModelsFactory.inject_submodules.<locals>.wrapper(*args, **kwargs)
     72 modules_kwargs = self.get_kwargs()
     73 new_kwargs = dict(list(kwargs.items()) + list(modules_kwargs.items()))
---> 74 return func(*args, **new_kwargs)

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/classification_models_3D/models/resnet.py:314, in ResNet34(input_shape, input_tensor, weights, classes, include_top, **kwargs)
    313 def ResNet34(input_shape=None, input_tensor=None, weights=None, classes=1000, include_top=True, **kwargs):
--> 314     return ResNet(
    315         MODELS_PARAMS['resnet34'],
    316         input_shape=input_shape,
    317         input_tensor=input_tensor,
    318         include_top=include_top,
    319         classes=classes,
    320         weights=weights,
    321         **kwargs
    322     )

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/classification_models_3D/models/resnet.py:231, in ResNet(model_params, input_shape, input_tensor, include_top, classes, weights, **kwargs)
    228 init_filters = 64
    230 # resnet bottom
--> 231 x = layers.BatchNormalization(name='bn_data', **no_scale_bn_params)(img_input)
    232 x = layers.ZeroPadding3D(padding=(3, 3, 3))(x)
    233 x = layers.Conv3D(init_filters, (7, 7, 7), strides=(2, 2, 2), name='conv0', **conv_params)(x)

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:925, in Layer.__call__(self, *args, **kwargs)
    919 # Functional Model construction mode is invoked when `Layer`s are called on
    920 # symbolic `KerasTensor`s, i.e.:
    921 # >> inputs = tf.keras.Input(10)
    922 # >> outputs = MyLayer()(inputs)  # Functional construction mode.
    923 # >> model = tf.keras.Model(inputs, outputs)
    924 if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
--> 925   return self._functional_construction_call(inputs, args, kwargs,
    926                                             input_list)
    928 # Maintains info about the `Layer.call` stack.
    929 call_context = base_layer_utils.call_context()

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:1098, in Layer._functional_construction_call(self, inputs, args, kwargs, input_list)
   1094 # Use `self._name_scope()` to avoid auto-incrementing the name.
   1095 with graph.as_default(), backend.name_scope(self._name_scope()):
   1096   # Build layer if applicable (if the `build` method has been
   1097   # overridden).
-> 1098   self._maybe_build(inputs)
   1099   cast_inputs = self._maybe_cast_inputs(inputs, input_list)
   1101   if not self.dynamic:
   1102     # Wrapping `call` function in autograph to allow for dynamic control
   1103     # flow and control dependencies in call. We are limiting this to
   (...)
   1106     # tf_convert will respect the value of autograph setting in the
   1107     # enclosing tf.function, if any.

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:2643, in Layer._maybe_build(self, inputs)
   2638 if not hasattr(self.build, '_is_default'):
   2639   # Any setup work performed only once should happen in an `init_scope`
   2640   # to avoid creating symbolic Tensors that will later pollute any eager
   2641   # operations.
   2642   with tf_utils.maybe_init_scope(self):
-> 2643     self.build(input_shapes)  # pylint:disable=not-callable
   2644 # We must set also ensure that the layer is marked as built, and the build
   2645 # shape is stored since user defined build functions may not be calling
   2646 # `super.build()`
   2647 Layer.build(self, input_shapes)

File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/layers/normalization.py:301, in BatchNormalizationBase.build(self, input_shape)
    299 for x in self.axis:
    300   if x < 0 or x >= ndims:
--> 301     raise ValueError('Invalid axis: %d' % x)
    302 if len(self.axis) != len(set(self.axis)):
    303   raise ValueError('Duplicate axis: %s' % self.axis)

ValueError: Invalid axis: 4

Looks pretty similar to the original, coming from keras normalization.py.

$ pip show keras
WARNING: Package(s) not found: keras
$ pip show tensorflow
Name: tensorflow
Version: 2.3.2
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /home/atom/.miniconda/envs/eece571f/lib/python3.8/site-packages
Requires: absl-py, astunparse, gast, google-pasta, grpcio, h5py, keras-preprocessing, numpy, opt-einsum, protobuf, six, tensorboard, tensorflow-estimator, termcolor, wheel, wrapt
Required-by: segmentation-models-3D

ZFTurbo · 2022-03-06T16:53:32Z

Can you also print your versions of:

keras_applications>=1.0.8
classification_models_3D>=1.0.3
efficientnet_3D>=1.0.2

nathanielatom · 2022-03-06T17:22:36Z

keras_applications==1.0.8
classification_models_3D==1.0.3
efficientnet_3D==1.0.2

ZFTurbo · 2022-03-06T17:25:04Z

Can you try this:

from classification_models_3D.tfkeras import Classifiers
ResNet18, preprocess_input = Classifiers.get('resnet18')
model = ResNet18(input_shape=(128, 128, 128, 3), weights='imagenet')

nathanielatom · 2022-03-07T20:02:13Z

Yep that works!

In [1]: from classification_models_3D.tfkeras import Classifiers
   ...: ResNet18, preprocess_input = Classifiers.get('resnet18')
   ...: model = ResNet18(input_shape=(128, 128, 128, 3), weights='imagenet')
2022-03-07 14:58:02.964417: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2022-03-07 14:58:02.964442: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-03-07 14:58:03.719302: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2022-03-07 14:58:03.742182: E tensorflow/stream_executor/cuda/cuda_driver.cc:314] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2022-03-07 14:58:03.742302: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: ||||||||||||||
2022-03-07 14:58:03.742374: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: ||||||||||||||
2022-03-07 14:58:03.742487: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 510.47.3
2022-03-07 14:58:03.742571: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 510.47.3
2022-03-07 14:58:03.742639: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 510.47.3
2022-03-07 14:58:03.742835: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-03-07 14:58:03.753121: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 1599960000 Hz
2022-03-07 14:58:03.753790: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x561f279e0040 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-03-07 14:58:03.753805: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Downloading data from https://github.com/ZFTurbo/classification_models_3D/releases/download/v1.0/resnet18_inp_channel_3_tch_0_top_True.h5
135127040/135126440 [==============================] - 13s 0us/step

karllandheer · 2022-07-05T20:52:49Z

Hello, your example on the main page doesn't show you have to put in the input_shape, however when I do not specify this I get an error. If you do indeed need to specify the input_shape please update the example for future users :)

abdulwaheedsoudagar · 2023-04-16T17:31:55Z

It was resolved by adding

`tf.keras.backend.set_image_data_format('channels_last')`

before importing unet. Also, do not import keras, just use tensorflow.

remyamanikantan · 2023-09-13T00:08:56Z

ModuleNotFoundError: No module named 'keras.engine' when importing segmentation_models_3D

navyanshmahla · 2023-09-24T09:04:56Z

ModuleNotFoundError: No module named 'keras.engine' when importing segmentation_models_3D

I also had the same error while I was using tensorflow==2.13.0
I rolled back to version 2.12.0 and it's working all fine for me. This is probably because Tensorflow 2.13.0 changed how we import keras modules which are not part of public APIs. In these cases direct imports won't work. keras.engine is not part of the public API.

There can be two solutions to this: Either roll back to a previous version of tensorflow (I rolled back to 2.12.0) or do the following:
I'm stating this from the official git repository of tensorflow releases:

If you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline: The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version.

segmentation-models-3D uses classification-models-3D where you can easily see that it uses keras.engine. Sadly, that hasn't still been changed for newer tf versions. So the best way is to roll back to older versions of tensorflow

remyamanikantan · 2023-09-24T10:59:10Z

Thank you so much for the solution.

…

On Sun, Sep 24, 2023, 10:05 AM Navyansh Mahla ***@***.***> wrote: ModuleNotFoundError: No module named 'keras.engine' when importing segmentation_models_3D I also had the same error while I was using tensorflow==2.13.0 I rolled back to version 2.12.0 and it's working all fine for me. This is probably because Tensorflow 2.13.0 changed how we import keras modules which are not part of public APIs. In these cases direct imports won't work. keras.engine is not part of the public API. There can be two solutions to this: Either roll back to a previous version of tensorflow (I rolled back to 2.12.0) or do the following: I'm stating this from the official git repository of tensorflow releases <https://github.com/tensorflow/tensorflow/releases/tag/v2.13.0>: If you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline: - The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version. segmentation-models-3D uses classification-models-3D where you can easily see that it uses keras.engine. Sadly, that hasn't still been changed for newer tf versions. So the best way is to roll back to previous versions. — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZ34NESUKIQZRSSJG4A4UK3X37ZUHANCNFSM5MLNEDQA> . You are receiving this because you commented.Message ID: ***@***.***>

ZFTurbo mentioned this issue Jan 20, 2022

not working for tensorflow 2x #5

Open

CarlosNacher mentioned this issue Aug 5, 2022

ValueError: axes don't match array #16

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to create basic model #8

Unable to create basic model #8

karllandheer commented Jan 20, 2022

karllandheer commented Jan 20, 2022

ZFTurbo commented Jan 20, 2022 •

edited

karllandheer commented Jan 20, 2022

karllandheer commented Jan 20, 2022

ZFTurbo commented Jan 21, 2022

karllandheer commented Jan 31, 2022

ZFTurbo commented Jan 31, 2022

nathanielatom commented Mar 5, 2022

ZFTurbo commented Mar 5, 2022

nathanielatom commented Mar 5, 2022

ZFTurbo commented Mar 6, 2022

nathanielatom commented Mar 6, 2022

ZFTurbo commented Mar 6, 2022

nathanielatom commented Mar 7, 2022

karllandheer commented Jul 5, 2022

abdulwaheedsoudagar commented Apr 16, 2023

remyamanikantan commented Sep 13, 2023

navyanshmahla commented Sep 24, 2023 •

edited

remyamanikantan commented Sep 24, 2023 via email

Unable to create basic model #8

Unable to create basic model #8

Comments

karllandheer commented Jan 20, 2022

karllandheer commented Jan 20, 2022

ZFTurbo commented Jan 20, 2022 • edited

karllandheer commented Jan 20, 2022

karllandheer commented Jan 20, 2022

ZFTurbo commented Jan 21, 2022

karllandheer commented Jan 31, 2022

ZFTurbo commented Jan 31, 2022

nathanielatom commented Mar 5, 2022

ZFTurbo commented Mar 5, 2022

nathanielatom commented Mar 5, 2022

ZFTurbo commented Mar 6, 2022

nathanielatom commented Mar 6, 2022

ZFTurbo commented Mar 6, 2022

nathanielatom commented Mar 7, 2022

karllandheer commented Jul 5, 2022

abdulwaheedsoudagar commented Apr 16, 2023

remyamanikantan commented Sep 13, 2023

navyanshmahla commented Sep 24, 2023 • edited

remyamanikantan commented Sep 24, 2023 via email

ZFTurbo commented Jan 20, 2022 •

edited

navyanshmahla commented Sep 24, 2023 •

edited