Incorrect output dimension on ios

[l0stpenguin]

I tried reconverting the latest pretrained model from the Pydnet repository to ios via tfcoreml. The conversion is successful but the output shape has wrong dimensions:
(1, 512, 256, 2)
I expected it to be 1 instead of 2. I know there is already a provided ios coreml file here but i plan to retrain the Pydnet model on my own dataset later. That's why i am attempting to do the conversation myself.

mlmodel = tfcoreml.convert(
            tf_model_path = './checkpoint/IROS18/frozen_model.pb',
            mlmodel_path = './checkpoint/IROS18/pydnet.mlmodel',
            output_feature_names = ['model/L0/ResizeBilinear'],
            image_input_names = ['input'],
            input_name_shape_dict = {'input':[1,512,256,3]},
            minimum_ios_deployment_target = '13'
 )

@GZaccaroni did you encounter such issue when you did the conversion for the ios part? Due to the incorrect dimension, i am not able to transform it to a valid image.
Here is the full conversion log from tfcoreml:

[SSAConverter] Converting function main ...
[SSAConverter] [1/143] Converting op type: 'Placeholder', name: 'input', output_shape: (1, 512, 256, 3).
[SSAConverter] [2/143] Converting op type: 'Const', name: 'model/pyramid/conv1a/mul/x'.
[SSAConverter] [3/143] Converting op type: 'Const', name: 'model/pyramid/conv1b/mul/x'.
[SSAConverter] [4/143] Converting op type: 'Const', name: 'model/pyramid/conv2a/mul/x'.
[SSAConverter] [5/143] Converting op type: 'Const', name: 'model/pyramid/conv2b/mul/x'.
[SSAConverter] [6/143] Converting op type: 'Const', name: 'model/pyramid/conv3a/mul/x'.
[SSAConverter] [7/143] Converting op type: 'Const', name: 'model/pyramid/conv3b/mul/x'.
[SSAConverter] [8/143] Converting op type: 'Const', name: 'model/pyramid/conv4a/mul/x'.
[SSAConverter] [9/143] Converting op type: 'Const', name: 'model/pyramid/conv4b/mul/x'.
[SSAConverter] [10/143] Converting op type: 'Const', name: 'model/pyramid/conv5a/mul/x'.
[SSAConverter] [11/143] Converting op type: 'Const', name: 'model/pyramid/conv5b/mul/x'.
[SSAConverter] [12/143] Converting op type: 'Const', name: 'model/pyramid/conv6a/mul/x'.
[SSAConverter] [13/143] Converting op type: 'Const', name: 'model/pyramid/conv6b/mul/x'.
[SSAConverter] [14/143] Converting op type: 'Const', name: 'model/L6/estimator/disp-3/mul/x'.
[SSAConverter] [15/143] Converting op type: 'Const', name: 'model/L6/estimator/disp-4/mul/x'.
[SSAConverter] [16/143] Converting op type: 'Const', name: 'model/L6/estimator/disp-5/mul/x'.
[SSAConverter] [17/143] Converting op type: 'Const', name: 'model/L6/upsampler/weights/read', output_shape: (2, 2, 8, 8).
[SSAConverter] [18/143] Converting op type: 'Const', name: 'model/L6/upsampler/biases/read', output_shape: (1, 8, 1, 1).
[SSAConverter] [19/143] Converting op type: 'Const', name: 'model/L6/upsampler/conv2d_transpose/output_shape', output_shape: (4,).
[SSAConverter] [20/143] Converting op type: 'Const', name: 'model/L6/upsampler/mul/x'.
[SSAConverter] [21/143] Converting op type: 'Const', name: 'model/L5/estimator/concat/axis'.
[SSAConverter] [22/143] Converting op type: 'Const', name: 'model/L5/estimator/disp-3/mul/x'.
[SSAConverter] [23/143] Converting op type: 'Const', name: 'model/L5/estimator/disp-4/mul/x'.
[SSAConverter] [24/143] Converting op type: 'Const', name: 'model/L5/estimator/disp-5/mul/x'.
[SSAConverter] [25/143] Converting op type: 'Const', name: 'model/L5/upsampler/weights/read', output_shape: (2, 2, 8, 8).
[SSAConverter] [26/143] Converting op type: 'Const', name: 'model/L5/upsampler/biases/read', output_shape: (1, 8, 1, 1).
[SSAConverter] [27/143] Converting op type: 'Const', name: 'model/L5/upsampler/conv2d_transpose/output_shape', output_shape: (4,).
[SSAConverter] [28/143] Converting op type: 'Const', name: 'model/L5/upsampler/mul/x'.
[SSAConverter] [29/143] Converting op type: 'Const', name: 'model/L4/estimator/concat/axis'.
[SSAConverter] [30/143] Converting op type: 'Const', name: 'model/L4/estimator/disp-3/mul/x'.
[SSAConverter] [31/143] Converting op type: 'Const', name: 'model/L4/estimator/disp-4/mul/x'.
[SSAConverter] [32/143] Converting op type: 'Const', name: 'model/L4/estimator/disp-5/mul/x'.
[SSAConverter] [33/143] Converting op type: 'Const', name: 'model/L4/upsampler/weights/read', output_shape: (2, 2, 8, 8).
[SSAConverter] [34/143] Converting op type: 'Const', name: 'model/L4/upsampler/biases/read', output_shape: (1, 8, 1, 1).
[SSAConverter] [35/143] Converting op type: 'Const', name: 'model/L4/upsampler/conv2d_transpose/output_shape', output_shape: (4,).
[SSAConverter] [36/143] Converting op type: 'Const', name: 'model/L4/upsampler/mul/x'.
[SSAConverter] [37/143] Converting op type: 'Const', name: 'model/L3/estimator/concat/axis'.
[SSAConverter] [38/143] Converting op type: 'Const', name: 'model/L3/estimator/disp-3/mul/x'.
[SSAConverter] [39/143] Converting op type: 'Const', name: 'model/L3/estimator/disp-4/mul/x'.
[SSAConverter] [40/143] Converting op type: 'Const', name: 'model/L3/estimator/disp-5/mul/x'.
[SSAConverter] [41/143] Converting op type: 'Const', name: 'model/L3/estimator/Slice/begin', output_shape: (4,).
[SSAConverter] [42/143] Converting op type: 'Const', name: 'model/L3/estimator/Slice/size', output_shape: (4,).
[SSAConverter] [43/143] Converting op type: 'Const', name: 'model/L3/estimator/mul/x'.
[SSAConverter] [44/143] Converting op type: 'Const', name: 'model/L0/size', output_shape: (2,).
[SSAConverter] [45/143] Converting op type: 'Transpose', name: 'input_to_nchw', output_shape: (1, 3, 512, 256).
[SSAConverter] [46/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv1a/Conv2D', output_shape: (1, 16, 256, 128).
[SSAConverter] [47/143] Converting op type: 'Mul', name: 'model/pyramid/conv1a/mul', output_shape: (1, 16, 256, 128).
[SSAConverter] [48/143] Converting op type: 'Maximum', name: 'model/pyramid/conv1a/Maximum', output_shape: (1, 16, 256, 128).
[SSAConverter] [49/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv1b/Conv2D', output_shape: (1, 16, 256, 128).
[SSAConverter] [50/143] Converting op type: 'Mul', name: 'model/pyramid/conv1b/mul', output_shape: (1, 16, 256, 128).
[SSAConverter] [51/143] Converting op type: 'Maximum', name: 'model/pyramid/conv1b/Maximum', output_shape: (1, 16, 256, 128).
[SSAConverter] [52/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv2a/Conv2D', output_shape: (1, 32, 128, 64).
[SSAConverter] [53/143] Converting op type: 'Mul', name: 'model/pyramid/conv2a/mul', output_shape: (1, 32, 128, 64).
[SSAConverter] [54/143] Converting op type: 'Maximum', name: 'model/pyramid/conv2a/Maximum', output_shape: (1, 32, 128, 64).
[SSAConverter] [55/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv2b/Conv2D', output_shape: (1, 32, 128, 64).
[SSAConverter] [56/143] Converting op type: 'Mul', name: 'model/pyramid/conv2b/mul', output_shape: (1, 32, 128, 64).
[SSAConverter] [57/143] Converting op type: 'Maximum', name: 'model/pyramid/conv2b/Maximum', output_shape: (1, 32, 128, 64).
[SSAConverter] [58/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv3a/Conv2D', output_shape: (1, 64, 64, 32).
[SSAConverter] [59/143] Converting op type: 'Mul', name: 'model/pyramid/conv3a/mul', output_shape: (1, 64, 64, 32).
[SSAConverter] [60/143] Converting op type: 'Maximum', name: 'model/pyramid/conv3a/Maximum', output_shape: (1, 64, 64, 32).
[SSAConverter] [61/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv3b/Conv2D', output_shape: (1, 64, 64, 32).
[SSAConverter] [62/143] Converting op type: 'Mul', name: 'model/pyramid/conv3b/mul', output_shape: (1, 64, 64, 32).
[SSAConverter] [63/143] Converting op type: 'Maximum', name: 'model/pyramid/conv3b/Maximum', output_shape: (1, 64, 64, 32).
[SSAConverter] [64/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv4a/Conv2D', output_shape: (1, 96, 32, 16).
[SSAConverter] [65/143] Converting op type: 'Mul', name: 'model/pyramid/conv4a/mul', output_shape: (1, 96, 32, 16).
[SSAConverter] [66/143] Converting op type: 'Maximum', name: 'model/pyramid/conv4a/Maximum', output_shape: (1, 96, 32, 16).
[SSAConverter] [67/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv4b/Conv2D', output_shape: (1, 96, 32, 16).
[SSAConverter] [68/143] Converting op type: 'Mul', name: 'model/pyramid/conv4b/mul', output_shape: (1, 96, 32, 16).
[SSAConverter] [69/143] Converting op type: 'Maximum', name: 'model/pyramid/conv4b/Maximum', output_shape: (1, 96, 32, 16).
[SSAConverter] [70/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv5a/Conv2D', output_shape: (1, 128, 16, 8).
[SSAConverter] [71/143] Converting op type: 'Mul', name: 'model/pyramid/conv5a/mul', output_shape: (1, 128, 16, 8).
[SSAConverter] [72/143] Converting op type: 'Maximum', name: 'model/pyramid/conv5a/Maximum', output_shape: (1, 128, 16, 8).
[SSAConverter] [73/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv5b/Conv2D', output_shape: (1, 128, 16, 8).
[SSAConverter] [74/143] Converting op type: 'Mul', name: 'model/pyramid/conv5b/mul', output_shape: (1, 128, 16, 8).
[SSAConverter] [75/143] Converting op type: 'Maximum', name: 'model/pyramid/conv5b/Maximum', output_shape: (1, 128, 16, 8).
[SSAConverter] [76/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv6a/Conv2D', output_shape: (1, 192, 8, 4).
[SSAConverter] [77/143] Converting op type: 'Mul', name: 'model/pyramid/conv6a/mul', output_shape: (1, 192, 8, 4).
[SSAConverter] [78/143] Converting op type: 'Maximum', name: 'model/pyramid/conv6a/Maximum', output_shape: (1, 192, 8, 4).
[SSAConverter] [79/143] Converting op type: 'Conv2D', name: 'model/pyramid/conv6b/Conv2D', output_shape: (1, 192, 8, 4).
[SSAConverter] [80/143] Converting op type: 'Mul', name: 'model/pyramid/conv6b/mul', output_shape: (1, 192, 8, 4).
[SSAConverter] [81/143] Converting op type: 'Maximum', name: 'model/pyramid/conv6b/Maximum', output_shape: (1, 192, 8, 4).
[SSAConverter] [82/143] Converting op type: 'Conv2D', name: 'model/L6/estimator/disp-3/Conv2D', output_shape: (1, 96, 8, 4).
[SSAConverter] [83/143] Converting op type: 'Mul', name: 'model/L6/estimator/disp-3/mul', output_shape: (1, 96, 8, 4).
[SSAConverter] [84/143] Converting op type: 'Maximum', name: 'model/L6/estimator/disp-3/Maximum', output_shape: (1, 96, 8, 4).
[SSAConverter] [85/143] Converting op type: 'Conv2D', name: 'model/L6/estimator/disp-4/Conv2D', output_shape: (1, 64, 8, 4).
[SSAConverter] [86/143] Converting op type: 'Mul', name: 'model/L6/estimator/disp-4/mul', output_shape: (1, 64, 8, 4).
[SSAConverter] [87/143] Converting op type: 'Maximum', name: 'model/L6/estimator/disp-4/Maximum', output_shape: (1, 64, 8, 4).
[SSAConverter] [88/143] Converting op type: 'Conv2D', name: 'model/L6/estimator/disp-5/Conv2D', output_shape: (1, 32, 8, 4).
[SSAConverter] [89/143] Converting op type: 'Mul', name: 'model/L6/estimator/disp-5/mul', output_shape: (1, 32, 8, 4).
[SSAConverter] [90/143] Converting op type: 'Maximum', name: 'model/L6/estimator/disp-5/Maximum', output_shape: (1, 32, 8, 4).
[SSAConverter] [91/143] Converting op type: 'Conv2D', name: 'model/L6/estimator/disp-6/Conv2D', output_shape: (1, 8, 8, 4).
[SSAConverter] [92/143] Converting op type: 'Conv2DBackpropInput', name: 'model/L6/upsampler/conv2d_transpose', output_shape: (1, 8, 16, 8).
[SSAConverter] [93/143] Converting op type: 'BiasAdd', name: 'model/L6/upsampler/BiasAdd', output_shape: (1, 8, 16, 8).
[SSAConverter] [94/143] Converting op type: 'Mul', name: 'model/L6/upsampler/mul', output_shape: (1, 8, 16, 8).
[SSAConverter] [95/143] Converting op type: 'Maximum', name: 'model/L6/upsampler/Maximum', output_shape: (1, 8, 16, 8).
[SSAConverter] [96/143] Converting op type: 'ConcatV2', name: 'model/L5/estimator/concat', output_shape: (1, 136, 16, 8).
[SSAConverter] [97/143] Converting op type: 'Conv2D', name: 'model/L5/estimator/disp-3/Conv2D', output_shape: (1, 96, 16, 8).
[SSAConverter] [98/143] Converting op type: 'Mul', name: 'model/L5/estimator/disp-3/mul', output_shape: (1, 96, 16, 8).
[SSAConverter] [99/143] Converting op type: 'Maximum', name: 'model/L5/estimator/disp-3/Maximum', output_shape: (1, 96, 16, 8).
[SSAConverter] [100/143] Converting op type: 'Conv2D', name: 'model/L5/estimator/disp-4/Conv2D', output_shape: (1, 64, 16, 8).
[SSAConverter] [101/143] Converting op type: 'Mul', name: 'model/L5/estimator/disp-4/mul', output_shape: (1, 64, 16, 8).
[SSAConverter] [102/143] Converting op type: 'Maximum', name: 'model/L5/estimator/disp-4/Maximum', output_shape: (1, 64, 16, 8).
[SSAConverter] [103/143] Converting op type: 'Conv2D', name: 'model/L5/estimator/disp-5/Conv2D', output_shape: (1, 32, 16, 8).
[SSAConverter] [104/143] Converting op type: 'Mul', name: 'model/L5/estimator/disp-5/mul', output_shape: (1, 32, 16, 8).
[SSAConverter] [105/143] Converting op type: 'Maximum', name: 'model/L5/estimator/disp-5/Maximum', output_shape: (1, 32, 16, 8).
[SSAConverter] [106/143] Converting op type: 'Conv2D', name: 'model/L5/estimator/disp-6/Conv2D', output_shape: (1, 8, 16, 8).
[SSAConverter] [107/143] Converting op type: 'Conv2DBackpropInput', name: 'model/L5/upsampler/conv2d_transpose', output_shape: (1, 8, 32, 16).
[SSAConverter] [108/143] Converting op type: 'BiasAdd', name: 'model/L5/upsampler/BiasAdd', output_shape: (1, 8, 32, 16).
[SSAConverter] [109/143] Converting op type: 'Mul', name: 'model/L5/upsampler/mul', output_shape: (1, 8, 32, 16).
[SSAConverter] [110/143] Converting op type: 'Maximum', name: 'model/L5/upsampler/Maximum', output_shape: (1, 8, 32, 16).
[SSAConverter] [111/143] Converting op type: 'ConcatV2', name: 'model/L4/estimator/concat', output_shape: (1, 104, 32, 16).
[SSAConverter] [112/143] Converting op type: 'Conv2D', name: 'model/L4/estimator/disp-3/Conv2D', output_shape: (1, 96, 32, 16).
[SSAConverter] [113/143] Converting op type: 'Mul', name: 'model/L4/estimator/disp-3/mul', output_shape: (1, 96, 32, 16).
[SSAConverter] [114/143] Converting op type: 'Maximum', name: 'model/L4/estimator/disp-3/Maximum', output_shape: (1, 96, 32, 16).
[SSAConverter] [115/143] Converting op type: 'Conv2D', name: 'model/L4/estimator/disp-4/Conv2D', output_shape: (1, 64, 32, 16).
[SSAConverter] [116/143] Converting op type: 'Mul', name: 'model/L4/estimator/disp-4/mul', output_shape: (1, 64, 32, 16).
[SSAConverter] [117/143] Converting op type: 'Maximum', name: 'model/L4/estimator/disp-4/Maximum', output_shape: (1, 64, 32, 16).
[SSAConverter] [118/143] Converting op type: 'Conv2D', name: 'model/L4/estimator/disp-5/Conv2D', output_shape: (1, 32, 32, 16).
[SSAConverter] [119/143] Converting op type: 'Mul', name: 'model/L4/estimator/disp-5/mul', output_shape: (1, 32, 32, 16).
[SSAConverter] [120/143] Converting op type: 'Maximum', name: 'model/L4/estimator/disp-5/Maximum', output_shape: (1, 32, 32, 16).
[SSAConverter] [121/143] Converting op type: 'Conv2D', name: 'model/L4/estimator/disp-6/Conv2D', output_shape: (1, 8, 32, 16).
[SSAConverter] [122/143] Converting op type: 'Conv2DBackpropInput', name: 'model/L4/upsampler/conv2d_transpose', output_shape: (1, 8, 64, 32).
[SSAConverter] [123/143] Converting op type: 'BiasAdd', name: 'model/L4/upsampler/BiasAdd', output_shape: (1, 8, 64, 32).
[SSAConverter] [124/143] Converting op type: 'Mul', name: 'model/L4/upsampler/mul', output_shape: (1, 8, 64, 32).
[SSAConverter] [125/143] Converting op type: 'Maximum', name: 'model/L4/upsampler/Maximum', output_shape: (1, 8, 64, 32).
[SSAConverter] [126/143] Converting op type: 'ConcatV2', name: 'model/L3/estimator/concat', output_shape: (1, 72, 64, 32).
[SSAConverter] [127/143] Converting op type: 'Conv2D', name: 'model/L3/estimator/disp-3/Conv2D', output_shape: (1, 96, 64, 32).
[SSAConverter] [128/143] Converting op type: 'Mul', name: 'model/L3/estimator/disp-3/mul', output_shape: (1, 96, 64, 32).
[SSAConverter] [129/143] Converting op type: 'Maximum', name: 'model/L3/estimator/disp-3/Maximum', output_shape: (1, 96, 64, 32).
[SSAConverter] [130/143] Converting op type: 'Conv2D', name: 'model/L3/estimator/disp-4/Conv2D', output_shape: (1, 64, 64, 32).
[SSAConverter] [131/143] Converting op type: 'Mul', name: 'model/L3/estimator/disp-4/mul', output_shape: (1, 64, 64, 32).
[SSAConverter] [132/143] Converting op type: 'Maximum', name: 'model/L3/estimator/disp-4/Maximum', output_shape: (1, 64, 64, 32).
[SSAConverter] [133/143] Converting op type: 'Conv2D', name: 'model/L3/estimator/disp-5/Conv2D', output_shape: (1, 32, 64, 32).
[SSAConverter] [134/143] Converting op type: 'Mul', name: 'model/L3/estimator/disp-5/mul', output_shape: (1, 32, 64, 32).
[SSAConverter] [135/143] Converting op type: 'Maximum', name: 'model/L3/estimator/disp-5/Maximum', output_shape: (1, 32, 64, 32).
[SSAConverter] [136/143] Converting op type: 'Conv2D', name: 'model/L3/estimator/disp-6/Conv2D', output_shape: (1, 8, 64, 32).
[SSAConverter] [137/143] Converting op type: 'Transpose', name: 'model/L3/estimator/disp-6/Conv2D_to_nhwc', output_shape: (1, 64, 32, 8).
[SSAConverter] [138/143] Converting op type: 'Slice', name: 'model/L3/estimator/Slice', output_shape: (1, 64, 32, 2).
[SSAConverter] [139/143] Converting op type: 'Sigmoid', name: 'model/L3/estimator/Sigmoid', output_shape: (1, 64, 32, 2).
[SSAConverter] [140/143] Converting op type: 'Mul', name: 'model/L3/estimator/mul', output_shape: (1, 64, 32, 2).
[SSAConverter] [141/143] Converting op type: 'Transpose', name: 'model/L3/estimator/mul_to_nchw', output_shape: (1, 2, 64, 32).
[SSAConverter] [142/143] Converting op type: 'ResizeBilinear', name: 'model/L0/ResizeBilinear_orig0', output_shape: (1, 2, 512, 256).
[SSAConverter] [143/143] Converting op type: 'Transpose', name: 'model/L0/ResizeBilinear', output_shape: (1, 512, 256, 2).

[FilippoAleotti]

Hi,
In the IROS2018 model the network predicts 2 disparities (ie, a left-aligned and a "virtual" right-aligned maps). If you are using this checkpoint, you have to slice the tensor

[l0stpenguin]

Yes i am using that checkpoint. I am not sure about how exactly i should proceed with slicing. I found only 1 output model/L0/ResizeBilinear and used that to freeze the graph. Are you suggesting that i should be modifying the output before freezing the graph?

[FilippoAleotti]

Yes, something like: tf.image.resize_images(self.disp2[:,:,:,0], size)

[l0stpenguin]

I am not experienced in neural networks, so i am quite confused. I don't manipulate any outputs nor do i run any inference in python. I just used the script in this post to export the pb file:

https://blog.metaflow.fr/tensorflow-how-to-freeze-a-model-and-serve-it-with-a-python-api-d4f3596b3adc

Since it's has only 1 output with 2 channels, i don't know how i am supposed to slice after freezing the graph.

[FilippoAleotti]

You have two options:

• edit the graph: by adding the lines I wrote before, you’re changing the graph. Doing so, you have to freeze this new graph
• keep the graph as it is: if you know that the predicted disparity has two channels, where just the first one is meaningful at testing time, you can slice the output in your app (that is, you keep just the first channel)