How to specify shape of input for TFLite model after receiving SavedModel format?

[Oktai15]

I use TF 2.0. Let's suppose I have a model that I successfully converted to SavedModel format. I used following code to convert my model in Imperative API (subclassed tf.keras.Model) to SavedModel:

tf.keras.experimental.export_saved_model(
    model, file_path,
    serving_only=True,
    input_signature=[tf.TensorSpec(shape=[None, None, None, 3], dtype=tf.float32)])

Now I want to convert my model to TFLite, but I have a problem, because I need to specify all dims except for batch. It's okay, but I want to do it after saving my model in SavedModel format. For example, if I want to do several models with different shapes, this feature is required (e.g, [None, 1280, 720, 3], [None, 600, 600, 3] etc.).

How can I do it in step where I convert my model to TFLite? I mean in this step (or something like that after receiving SavedModel format):

converter = tf.lite.TFLiteConverter.from_saved_model(file_path)
tflite_model = converter.convert()

Maybe I can specify input_shape in convert() or another way?

[gargn]

There is no option to specify input_shape in 2.0. You can reload your SavedModel yourself, set the input shape, and then call from_concrete_functions. Something similar to the following code (which hasn't been tested):

model = tf.saved_model.load(export_dir)
concrete_func = model.signatures[
  tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([None, 1280, 720, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])
...

[Oktai15]

@gargn oh, thanks, great workaround :)

[Oktai15]

We can leave this issue to be opened if you plan to add this ability to converter directly.

[gargn]

Adding this functionality would require the addition of input_arrays to the API. However, with the functions based focus in 2.0, users aren't expected to know their model's input arrays. We determined adding input_shapes to the API would introduce additional usability issues. Since the workaround is relatively simple, we decided that was the preferred approach for saved models with dynamic shapes.

We will be investigating how to improve support for dynamic shapes in TFLite which should eliminate the need for this parameter: #24607.

[yogithesymbian]

There is no option to specify input_shape in 2.0. You can reload your SavedModel yourself, set the input shape, and then call from_concrete_functions. Something similar to the following code (which hasn't been tested):

model = tf.saved_model.load(export_dir)
concrete_func = model.signatures[
  tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([None, 1280, 720, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])
...

i have train on here collab image classificatioin and have model.tflite, could you give some full tutorial to modified the shape until export the tflite ?

inception_v3_spec = model_spec.ImageModelSpec(
    uri='https://tfhub.dev/google/imagenet/inception_v3/feature_vector/1')
inception_v3_spec.input_image_shape = [299, 299] // this same like above ? but i got error buffersize when implementation to android (force close)

but in object detection guide

• Convert the frozen graph to the TFLite model.

tflite_convert \
  --input_shape=1,300,300,3 \
  --input_arrays=normalized_input_image_tensor \
  --output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 \
  --allow_custom_ops \
  --graph_def_file=exported_model/tflite_graph.pb \
  --output_file=<directory with the TensorFlow examples repository>/lite/examples/object_detection/android/app/src/main/assets/detect.tflite

input_shape=1,300,300,3 because the pretrained model works only with that input shape.

allow_custom_ops is necessary to allow TFLite_Detection_PostProcess operation.

input_arrays and output_arrays can be drawn from the visualized graph of the example detection model.

[hi-one-gg]

model = tf.saved_model.load(export_dir)
concrete_func = model.signatures[
tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([None, 1280, 720, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])

not working. It said,
ValueError: Dimension 1 in both shapes must be equal, but are 3 and 320. Shapes are [1,3,320,320] and [1,320,320,3].

[akl-01]

@blackCmd, Hello!
Did you solved this problem?

model = tf.saved_model.load(export_dir)
concrete_func = model.signatures[
tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([None, 1280, 720, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])

not working. It said, ValueError: Dimension 1 in both shapes must be equal, but are 3 and 320. Shapes are [1,3,320,320] and [1,320,320,3].