"Input shape axis 0 must equal 1" when using EfficientDet D0 on tensorflow-serving 2.3.0

[denisb411]

I train the model normally using model_main_tf2.py passing the pipeline_config_path using the ssd_efficientdet_d0_512x512_coco17_tpu-8.config pipeline config, which refers to the model of efficientdet d0.

It trains successfully as I used to do with faster_rcnn_resnet101 (tf 1.5).

I export the model using the script exporter_main_v2.py. It exports succesfully and it can be normally loaded by tensorflow-serving.

os.system(f'python -u exporter_main_v2.py \
            --input_type image_tensor \
            --pipeline_config_path={pipeline_file} \
            --trained_checkpoint_dir={training_dir} \
            --output_directory={serving_frozen_graph_dir}')

The error occurs when I try to inference batches from this model.
I execute the serving's grpc request with:

# tf-serving communication setup
options = [('grpc.max_receive_message_length', 100 * 4000 * 4000)]
channel = grpc.insecure_channel('{host}:{port}'.format(host='localhost', port=str(self.serving_grpc_port)), options = options)
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = 'ivi-detector'
request.model_spec.signature_name = 'serving_default'
request.inputs['input_tensor'].CopyFrom(tf.make_tensor_proto(imgs_array, shape=imgs_array.shape))

res = stub.Predict(request, 100.0)

boxes = np.reshape(res.outputs['detection_boxes'].float_val, [len(imgs_array), 100,4])
classes = np.reshape(res.outputs['detection_classes'].float_val, [len(imgs_array), 100])
scores = np.reshape(res.outputs['detection_scores'].float_val, [len(imgs_array), 100])

And it returns the following error:

debug_error_string = "{"created":"@1598292999.094988994","description":"Error received from peer ipv6:[::1]:8500","file":"src/core/lib/surface/call.cc","file_line":1061,"grpc_message":"2 root error(s) found.\n  (0) Invalid argument: Input shape axis 0 must equal 1, got shape [10,1560,840,3]\n\t [[{{node Preprocessor/unstack}}]]\n\t [[StatefulPartitionedCall/StatefulPartitionedCall/Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Reshape_7/_1463]]\n  (1) Invalid argument: Input shape axis 0 must equal 1, got shape [10,1560,840,3]\n\t [[{{node Preprocessor/unstack}}]]\n0 successful operations.\n0 derived errors ignored.","grpc_status":3}"

If I try to process using single images it works normally. I do it with:

image_expanded = np.expand_dims(imgs_array[0], axis=0)
request.inputs['input_tensor'].CopyFrom(tf.make_tensor_proto(image_expanded, shape=image_expanded.shape))
res = stub.Predict(request, 10.0)

boxes = np.reshape(res.outputs['detection_boxes'].float_val, [100,4])
classes = res.outputs['detection_classes'].float_val
scores = res.outputs['detection_scores'].float_val

Before started using efficientdet I was using faster_rcnn_resnet101_coco and it worked normally with both single images and batched images (I think that the shape=imgs_array.shape was doing the job).

Is this a bug or some configuration that I skipped when exporting the model?

[ktiwary2]

I think it is to do with the model itself. You can try to see the input shapes of the tensorflow model through saved_model_cli show --all --dir /path/to/saved/model/directory/
It will output something like this and you'll see that the batch dim is fixed to 1.

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['input_tensor'] tensor_info:
        dtype: DT_UINT8
        shape: (1, -1, -1, 3)
        name: serving_default_input_tensor:0

[sourabhx003]

The input axis 0 is the batch size so I tried running a (1,640,640,3) and it ran fine for me.