Error converting custom model to .tflite using Toco

[gmalsagov]

System information

• Have I written custom code: Yes, see neural net structure here
• OS Platform and Distribution: Mac OS X 10.13.4
• TensorFlow installed from (source or binary): pip (binary)
• TensorFlow version (use command below): 1.10.0
• Python version: 2.7.10
• Bazel version (if compiling from source): 0.15.2-homebrew
• GCC/Compiler version (if compiling from source): v1.10.0-rc1-19-g656e7a2b34
• CUDA/cuDNN version: N/A
• GPU model and memory: N/A
• Exact command to reproduce:
  toco
  --graph_def_file='path to frozen_model.pb'
  --input_format=TENSORFLOW_GRAPHDEF
  --output_format=TFLITE
  --inference_type=FLOAT
  --input_type=INT
  --input_arrays=input_x
  --output_arrays=output/predictions
  --input_shapes=1,78
  --output_file='path to model.tflite'

Describe the problem

I have a Tensorflow Convolutional Neural Network model for Multi-class Sentiment Analysis. I removed Dropout layer from the frozen model and it produces inference fine. When I am trying to convert it into a TfLite version using Toco I am getting an error that has something to do with dimensions of an input tensor (see error below). This tensor leads to a convolution tensor whose function takes a 4d input tensor.

2018-08-19 15:27:50.977853: F tensorflow/contrib/lite/toco/graph_transformations/propagate_fixed_sizes.cc:130] Check failed: input_shape.dimensions_count() == 4 Conv ops require 4D inputs. Input array "embedding/ExpandDims" is 3D.

Output from terminal:

`
2018-08-19 15:27:45.418932: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Traceback (most recent call last):
File "/tensorflow/venv/bin/toco", line 11, in
sys.exit(main())
File "/tensorflow/venv/lib/python2.7/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 370, in main
app.run(main=run_main, argv=sys.argv[:1])
File "/tensorflow/venv/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/tensorflow/venv/lib/python2.7/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 366, in run_main
_convert_model(tflite_flags)
File "/tensorflow/venv/lib/python2.7/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 143, in _convert_model
output_data = converter.convert()
File "/tensorflow/venv/lib/python2.7/site-packages/tensorflow/contrib/lite/python/lite.py", line 374, in convert
dump_graphviz_video=self.dump_graphviz_video)
File "/tensorflow/venv/lib/python2.7/site-packages/tensorflow/contrib/lite/python/convert.py", line 246, in toco_convert
input_data.SerializeToString())
File "/tensorflow/venv/lib/python2.7/site-packages/tensorflow/contrib/lite/python/convert.py", line 106, in toco_convert_protos
(stdout, stderr))
RuntimeError: TOCO failed see console for info.
2018-08-19 15:27:50.968669: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 40 operators, 57 arrays (0 quantized)
2018-08-19 15:27:50.975773: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 40 operators, 57 arrays (0 quantized)
2018-08-19 15:27:50.977853: F tensorflow/contrib/lite/toco/graph_transformations/propagate_fixed_sizes.cc:130] Check failed: input_shape.dimensions_count() == 4 Conv ops require 4D inputs. Input array "embedding/ExpandDims" is 3D.

None
`

Source code / logs

I am using tf.expand_dims command only when declaring embedding layer to expand tensor to 4d prior to feeding it into a tf.nn.conv2d function:

self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1) conv = tf.nn.conv2d(self.embedded_chars, W, strides=[1, 1, 1, 1], padding='VALID', name='conv')

Does this mean that tensorflow lite doesn't support expand dims yet? Would appreciate an explanation or any possible walkarounds for my issue.

Thanks

[gmalsagov]

Update:

I've replaced tf.nn.expand_dims() with alternative function

shape1 = self.embedded_chars.get_shape().as_list()
print(shape1)
dim = shape[1:]
dim = [-1] + dim + [1]
self.embedded_chars_expanded = tf.reshape(self.embedded_chars, dim)
shape2 = self.embedded_chars_expanded.get_shape().as_list()
print(shape2)

Both shapes are now (None, 40, 300, 1) as required. Model trains and does inference fine but during conversion to .tflite new error pops up.

RuntimeError: TOCO failed see console for info. 2018-08-19 22:13:20.177661: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 31 operators, 48 arrays (0 quantized) 2018-08-19 22:13:20.189193: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 31 operators, 48 arrays (0 quantized) 2018-08-19 22:13:20.192098: F tensorflow/contrib/lite/toco/graph_transformations/propagate_fixed_sizes.cc:442] Check failed: input_flat_size >= product_non_wildcard_dims (300 vs. 12000)Array not large enough to fill the requested dimensions for Reshape op with output "embedding/Reshape". Are your input shapes correct?

@aselle Would you please take a look at this?

[gargn]

Can you provide your model and a link to the code that's producing the model if possible? Any additional information will help in debugging.

[gmalsagov]

@gargn here is the link to my code as well as the frozen graph.

https://github.com/gmalsagov/Emotion-Multiclass-CNN/tree/dev2/cnn-embeddings
frozen_model.pb.zip

[gargn]

To answer your original question - ExpandDims is supported in TOCO when the input tensor has a static shape.

However, I am having trouble reproducing your issue from the frozen graph you provided. I tried running the command below on the tf-nightly build (installed via pip install tf-nightly) and I got an error stating that RandomUniform is not supported. When I added the --allow_custom_ops flag, TOCO converted the model without error.

tflite_convert \
--graph_def_file=$TENSORFLOW_FILE \
--input_arrays=input_x \
--output_arrays=output/predictions \
--input_shapes=1,40 \
--output_file=$TFLITE_FILE

A few changes I did have to make from your original command:

1. I removed --input_type=INT. The supported flags are listed here. It is possible you intended on using --inference_input_type. --inference_input_type=QUANTIZED_UINT8 is used for models where the input is quantized but the model is a float model.
2. I changed the input_shapes from 1,78 to 1,40.

Just to summarize, I was able to run the following command without TOCO reporting an error:

tflite_convert \
--graph_def_file=$TENSORFLOW_FILE \
--input_arrays=input_x \
--output_arrays=output/predictions \
--input_shapes=1,40 \
--output_file=$TFLITE_FILE \
--allow_custom_ops

Also, just as a note, tflite_convert is aliased to the same tool as toco. I use tflite_convert for my commands because that is what we use in our documentation.

Can you try running your model with the tf-nightly build instead of 1.10 and see if you get the same results?

[gmalsagov]

Thank you @gargn the issue has been solved when used tf-nightly instead of tensorflow 1.10.0

[gargn]

When running a model without the supported ops, the recommended steps are one of the following:

1. Add a custom op as described here.
2. Only use the ops that are currently supported by TFLite in your model.

Closing this issue. Feel free to reopen, or open a new bug, if you hit any other issues.

[HiIcy]

When running a model without the supported ops, the recommended steps are one of the following:

1. Add a custom op as described here.
2. Only use the ops that are currently supported by TFLite in your model.

Closing this issue. Feel free to reopen, or open a new bug, if you hit any other issues.

@gargn ,hello ,the link here is 404,can you resend?