Problem Converting tensorflow frozen model to onnx

[Rahul0985]

Describe the bug
conversion of tensorflow frozen model(BILSTM + CRF) to onnx failed .

Urgency
None

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): windows 10
• Tensorflow Version: 1.15
• Python version: 3.7

To Reproduce
Code(to convert (BILSTM + CRF) model to tf frozen model):
import os
import keras;
from keras import backend as K
import tensorflow.compat.v1 as tf
from keras.models import Model
from keras.layers import LSTM,Input, Embedding, Dense, TimeDistributed, Dropout, Bidirectional, Lambda
#from keras_contrib.layers.crf import CRF
from tensorflow.python.tools import freeze_graph
from keras_contrib.layers.crf import CRF, crf_loss, crf_viterbi_accuracy
#K.set_learning_phase(0);
print(keras.name)
def create_model():
maxlen = 100;
n_tags = 100
n_words = 100
input = Input(shape=(maxlen,), name="input")
model = Embedding(input_dim=n_words + 1, output_dim=250,
input_length=maxlen)(input)
model = Bidirectional(LSTM(units=150, return_sequences=True,
recurrent_dropout=0.1))(model) # variational biLSTM
model = TimeDistributed(Dense(150, activation="relu"))(model)
crf = CRF(n_tags) # CRF layer
out = crf(model) # output
model = Model(input, out)
model.compile(optimizer="rmsprop", loss=crf.loss_function, metrics=[crf.accuracy])
model.summary()
return model

model = create_model()

save_dir = "C:\Users\rahul\PycharmProjects\XYZ\model"

tf.saved_model.simple_save(K.get_session(),
save_dir,
inputs={"input": model.inputs[0]},
outputs={"output": model.outputs[0]}
)

freeze_graph.freeze_graph(None,
None,
None,
None,
model.outputs[0].op.name,
None,
None,
os.path.join(save_dir, "frozen_model.pb"),
False,
"",
input_saved_model_dir=save_dir)

...................................................................................................................................................................................................
Then i use
python -m tf2onnx.convert --input frozen_model.pb --output model.onnx --outputs crf_1/cond/Merge:0 --inputs input_1:0
to convert into onnx model.
Expected behavior
Successful conversion of model from tf to onnx

Screenshots
2020-03-30 00:41:37.689103: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_100.dll'; dlerror: cudart64_100.dll not found
2020-03-30 00:41:37.701017: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING:tensorflow:From C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\verbose_logging.py:72: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_ve
rbosity instead.

2020-03-30 00:41:41,006 - WARNING - From C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\verbose_logging.py:72: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.
v1.logging.set_verbosity instead.

2020-03-30 00:41:41.009392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-03-30 00:41:41.756002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1050 major: 6 minor: 1 memoryClockRate(GHz): 1.493
pciBusID: 0000:02:00.0
2020-03-30 00:41:41.767085: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_100.dll'; dlerror: cudart64_100.dll not found
2020-03-30 00:41:41.773644: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cublas64_100.dll'; dlerror: cublas64_100.dll not found
2020-03-30 00:41:41.782547: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cufft64_100.dll'; dlerror: cufft64_100.dll not found
2020-03-30 00:41:41.801789: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'curand64_100.dll'; dlerror: curand64_100.dll not found
2020-03-30 00:41:41.811840: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cusolver64_100.dll'; dlerror: cusolver64_100.dll not found
2020-03-30 00:41:41.830418: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cusparse64_100.dll'; dlerror: cusparse64_100.dll not found
2020-03-30 00:41:41.841328: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudnn64_7.dll'; dlerror: cudnn64_7.dll not found
2020-03-30 00:41:41.855089: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you woul
d like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2020-03-30 00:41:41.890716: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-03-30 00:41:41.909786: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-03-30 00:41:41.919578: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]
2020-03-30 00:41:42,547 - WARNING - inputs [input_1:0] is not in frozen graph, delete them
2020-03-30 00:41:42.886264: I tensorflow/tools/graph_transforms/transform_graph.cc:317] Applying fold_batch_norms
2020-03-30 00:41:42.906852: I tensorflow/tools/graph_transforms/transform_graph.cc:317] Applying fold_old_batch_norms
2020-03-30 00:41:43.283588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-03-30 00:41:43.292791: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]
2020-03-30 00:41:43,296 - INFO - Using tensorflow=1.15.0, onnx=1.6.0, tf2onnx=1.5.5/3c8f90
2020-03-30 00:41:43,296 - INFO - Using opset <onnx, 8>
2020-03-30 00:41:43,881 - ERROR - Tensorflow op [bidirectional_1/ReverseV2: ReverseV2] is not supported
2020-03-30 00:41:43,893 - ERROR - Tensorflow op [bidirectional_1/ReverseV2_1: ReverseV2] is not supported
2020-03-30 00:41:44,016 - ERROR - Tensorflow op [crf_1/ReverseV2: ReverseV2] is not supported
2020-03-30 00:41:44,067 - ERROR - Tensorflow op [crf_1/ReverseV2_1: ReverseV2] is not supported
2020-03-30 00:41:44,067 - ERROR - Failed to convert node crf_1/one_hot
OP=OneHot
Name=crf_1/one_hot
Inputs:
crf_1/Squeeze:0=Squeeze, [-1, 100], 6
crf_1/one_hot/depth:0=Const, [], 6
crf_1/one_hot/on_value:0=Const, [], 1
crf_1/one_hot/off_value:0=Const, [], 1
Outpus:
crf_1/one_hot:0=[-1, 100, 100], 1
Traceback (most recent call last):
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\tfonnx.py", line 356, in tensorflow_onnx_mapping
func(g, node, **kwargs)
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\onnx_opset\tensor.py", line 1053, in version_1
raise ValueError("onehot op: only rank1 is supported")
ValueError: onehot op: only rank1 is supported
2020-03-30 00:41:44,070 - ERROR - Unsupported ops: Counter({'ReverseV2': 4})
Traceback (most recent call last):
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\convert.py", line 161, in
main()
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\convert.py", line 145, in main
inputs_as_nchw=args.inputs_as_nchw)
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\tfonnx.py", line 579, in process_tf_graph
raise exceptions[0]
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\tfonnx.py", line 356, in tensorflow_onnx_mapping
func(g, node, **kwargs)
File "C:\Users\rahul\Anaconda3\envs\XYZ\lib\site-packages\tf2onnx\onnx_opset\tensor.py", line 1053, in version_1
raise ValueError("onehot op: only rank1 is supported")
ValueError: onehot op: only rank1 is supported

Additional context

[Rahul0985]

adding tf frozen model
frozen_model.zip

[jignparm]

@Rahul0985 -- sorry for the untimely reply. You should be able to use latest master of tf2onnx is able to convert your model successfully. See command line below -- using --inputs input:0 --opset 11

###log

python -m tf2onnx.convert --input frozen_model.pb --output model.onnx --outputs crf_1/cond/Merge:0 --inputs input:0 -- opset 11

2020-04-30 09:49:37,124 - INFO - Using tensorflow=1.14.0, onnx=1.6.0, tf2onnx=1.6.0/82f805
2020-04-30 09:49:37,124 - INFO - Using opset <onnx, 11>
2020-04-30 09:49:37,938 - INFO - Optimizing ONNX model
2020-04-30 09:49:39,080 - INFO - After optimization: And -1 (6->5), Cast -29 (61->32), Concat -1 (7->6), Const -153 (224->71), Expand -1 (6->5), Identity -27 (36->9), Less -3 (14->11), MatMul -1 (5->4), Mul -1 (12->11), ReduceSum -1 (2->1), Reshape -2 (6->4), Shape -4 (17->13), Slice -1 (28->27), Split -1 (2->1), Squeeze -7 (19->12), Tile -3 (4->1), Transpose -1 (15->14), Unsqueeze -10 (20->10)
2020-04-30 09:49:39,111 - INFO -
2020-04-30 09:49:39,111 - INFO - Successfully converted TensorFlow model frozen_model.pb to ONNX
2020-04-30 09:49:39,111 - INFO - ONNX model is saved at model.onnx

[guschmue]

I assume this is fixed.

[Aaryanverma]

@guschmue Hi, can you also tell how to convert above onnx converted model into tensorRT engine file successfully? I'm not able to convert because of a reverse sequence operation which is not supported by tensorRT yet. Can you help in this?

[guschmue]

don't think there is much we can do. Tried with tensorrt 7 ?

[Aaryanverma]

don't think there is much we can do. Tried with tensorrt 7 ?

Yes tried with tensorrr 7 but it doesn't support reverse sequence operation yet. That's where I'm stuck