Converting TF 2 Object Detection Model to TensorRT #9032
Description
Prerequisites
Please answer the following questions for yourself before submitting an issue.
- I am using the latest TensorFlow Model Garden release and TensorFlow 2.
- I am reporting the issue to the correct repository. (Model Garden official or research directory)
- I checked to make sure that this issue has not already been filed.
1. The entire URL of the file you are using
2. Describe the bug
Hey there! I am trying to convert the SSD ResNet50 V1 FPN 640x640 (RetinaNet50) of the new Tensorflow 2 Object Detection API to a TensorRT Model to run on my Jetson AGX Board.
3. Steps to reproduce
I am running the following Code:
import tensorflow as tf
import numpy as np
from tensorflow.python.compiler.tensorrt import trt_convert as trt
input_saved_model_dir = './ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/saved_model/'
output_saved_model_dir = './models/tensorRT/'
num_runs = 2
conversion_params = trt.DEFAULT_TRT_CONVERSION_PARAMS
conversion_params = conversion_params._replace(max_workspace_size_bytes=(1<<32))
conversion_params = conversion_params._replace(precision_mode="FP16")
# conversion_params = conversion_params._replace(maximum_cached_engiens=100)
converter = trt.TrtGraphConverterV2(input_saved_model_dir=input_saved_model_dir,conversion_params=conversion_params)
converter.convert()
def my_input_fn():
for _ in range(num_runs):
inp1 = np.random.normal(size=(1, 640, 640, 3)).astype(np.uint8)
yield inp1
converter.build(input_fn=my_input_fn)
converter.save(output_saved_model_dir)When running that Code I am getting the following Error:
InvalidArgumentError Traceback (most recent call last)
<ipython-input-7-d7c3941a6051> in <module>
7 yield inp1
8
----> 9 converter.build(input_fn=my_input_fn)
10 converter.save(output_saved_model_dir)
/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/compiler/tensorrt/trt_convert.py in build(self, input_fn)
1172 if not first_input:
1173 first_input = inp
-> 1174 func(*map(ops.convert_to_tensor, inp))
1175
1176 if self._need_trt_profiles:
/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
1603 TypeError: For invalid positional/keyword argument combinations.
1604 """
-> 1605 return self._call_impl(args, kwargs)
1606
1607 def _call_impl(self, args, kwargs, cancellation_manager=None):
/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _call_impl(self, args, kwargs, cancellation_manager)
1643 raise TypeError("Keyword arguments {} unknown. Expected {}.".format(
1644 list(kwargs.keys()), list(self._arg_keywords)))
-> 1645 return self._call_flat(args, self.captured_inputs, cancellation_manager)
1646
1647 def _filtered_call(self, args, kwargs):
/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
1744 # No tape is watching; skip to running the function.
1745 return self._build_call_outputs(self._inference_function.call(
-> 1746 ctx, args, cancellation_manager=cancellation_manager))
1747 forward_backward = self._select_forward_and_backward_functions(
1748 args,
/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
596 inputs=args,
597 attrs=attrs,
--> 598 ctx=ctx)
599 else:
600 outputs = execute.execute_with_cancellation(
/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:
InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Input shape axis 0 must equal 1, got shape [640,640,3]
[[node StatefulPartitionedCall/Preprocessor/unstack (defined at <ipython-input-7-d7c3941a6051>:2) ]]
[[StatefulPartitionedCall/Postprocessor/BatchMultiClassNonMaxSuppression/TRTEngineOp_11/_106]]
(1) Invalid argument: Input shape axis 0 must equal 1, got shape [640,640,3]
[[node StatefulPartitionedCall/Preprocessor/unstack (defined at <ipython-input-7-d7c3941a6051>:2) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_pruned_176027]
Function call stack:
pruned -> pruned
The error says that there is a Problem with my Input Dimensions. It specificially says that my first (index 0) dimension needs to be 1. I am passing a numpy array with the first Dimension beeing 1 (1, 640, 640, 3), but in the Error message it says that the Array is of shape [640,640,3].
I am not sure how to change my input so that it satisfies the requirements.
Thanks for your help!
4. Expected behavior
I want a TensorRT optimized Model so that I can run it on my Jetson AGX
6. System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
- TensorFlow version (use command below): 2.2
- Python version: 3.7