A whole lot of extra operations (including interpolate, 3d convs/deconvs/pooling)

[oliver-batchelor]

This is probably not merge-able generally as (some?) of it I think might use TensorRT 7 features, should there be a branch for TensorRT 7 and try to get torch2trt to support as many operations as possible?

[jaybdub]

Thanks for sharing!

Currently the unchecked TensorRT 7 requirement is a blocker for merging, since many of the projects using torch2trt still use TensorRT 5.

That said, I'd love to support these converters if we can add version checking and ensure all relevant test cases pass for TensorRT 5,6,7.

Best,
John

[jaybdub]

max_workspace_size can be provided as argument to torch2trt_kwargs where relevant when using @add_module_test

[Jockeypan]

I found a problem when converting pytorch's interpolate to trt's resize layer. The result just does not match. I tried your code and got the same result.
Point is nearest mode works fine, but linear does not work. I tested with trt6 and trt7, all got the same result.
@add_module_test(torch.float32, torch.device('cuda'), [(1, 1, 3, 3)])
def test_bilinear_mode():
return torch.nn.Upsample(scale_factor=4, mode="bilinear", align_corners=True)

I printed the input and outputs:
inputs:
(tensor([[[[ 0.8399, 0.1675, 0.2410],
[-1.1796, 0.1098, 2.2553],
[-0.3657, -0.9107, -0.4013]]]], device='cuda:0'),)
outputs:
tensor([[[[ 0.8399, 0.7176, 0.5954, 0.4731, 0.3509, 0.2286, 0.1742,
0.1875, 0.2009, 0.2143, 0.2276, 0.2410],
[ 0.4727, 0.4153, 0.3579, 0.3005, 0.2431, 0.1857, 0.1979,
0.2798, 0.3617, 0.4435, 0.5254, 0.6072],
[ 0.1055, 0.1130, 0.1204, 0.1279, 0.1353, 0.1428, 0.2217,
0.3721, 0.5224, 0.6728, 0.8231, 0.9735],
[-0.2616, -0.1893, -0.1170, -0.0447, 0.0276, 0.0999, 0.2455,
0.4643, 0.6832, 0.9020, 1.1209, 1.3397],
[-0.6288, -0.4917, -0.3545, -0.2173, -0.0802, 0.0570, 0.2692,
0.5566, 0.8439, 1.1312, 1.4186, 1.7059],
[-0.9960, -0.7940, -0.5920, -0.3900, -0.1879, 0.0141, 0.2930,
0.6488, 1.0047, 1.3605, 1.7163, 2.0722],
[-1.1056, -0.9015, -0.6974, -0.4932, -0.2891, -0.0850, 0.1986,
0.5616, 0.9247, 1.2877, 1.6507, 2.0138],
[-0.9576, -0.8141, -0.6707, -0.5272, -0.3837, -0.2402, -0.0140,
0.2949, 0.6039, 0.9128, 1.2218, 1.5307],
[-0.8096, -0.7268, -0.6440, -0.5611, -0.4783, -0.3955, -0.2266,
0.0283, 0.2831, 0.5380, 0.7929, 1.0477],
[-0.6617, -0.6395, -0.6173, -0.5951, -0.5729, -0.5507, -0.4392,
-0.2384, -0.0376, 0.1631, 0.3639, 0.5647],
[-0.5137, -0.5521, -0.5906, -0.6290, -0.6675, -0.7059, -0.6518,
-0.5051, -0.3584, -0.2117, -0.0650, 0.0817],
[-0.3657, -0.4648, -0.5639, -0.6630, -0.7621, -0.8612, -0.8644,
-0.7718, -0.6792, -0.5866, -0.4939, -0.4013]]]], device='cuda:0')
outputs_trt:
tensor([[[[ 0.8399, 0.6718, 0.5037, 0.3356, 0.1675, 0.1859, 0.2042,
0.2226, 0.2410, 0.2410, 0.2410, 0.2410],
[ 0.3350, 0.2895, 0.2441, 0.1986, 0.1531, 0.3009, 0.4488,
0.5967, 0.7446, 0.7446, 0.7446, 0.7446],
[-0.1698, -0.0927, -0.0156, 0.0615, 0.1387, 0.4160, 0.6934,
0.9708, 1.2481, 1.2481, 1.2481, 1.2481],
[-0.6747, -0.4750, -0.2752, -0.0755, 0.1242, 0.5311, 0.9380,
1.3448, 1.7517, 1.7517, 1.7517, 1.7517],
[-1.1796, -0.8572, -0.5349, -0.2125, 0.1098, 0.6462, 1.1826,
1.7189, 2.2553, 2.2553, 2.2553, 2.2553],
[-0.9761, -0.7684, -0.5607, -0.3530, -0.1453, 0.2888, 0.7229,
1.1570, 1.5911, 1.5911, 1.5911, 1.5911],
[-0.7727, -0.6796, -0.5865, -0.4935, -0.4004, -0.0686, 0.2633,
0.5951, 0.9270, 0.9270, 0.9270, 0.9270],
[-0.5692, -0.5908, -0.6124, -0.6340, -0.6556, -0.4260, -0.1964,
0.0332, 0.2628, 0.2628, 0.2628, 0.2628],
[-0.3657, -0.5020, -0.6382, -0.7745, -0.9107, -0.7834, -0.6560,
-0.5287, -0.4013, -0.4013, -0.4013, -0.4013],
[-0.3657, -0.5020, -0.6382, -0.7745, -0.9107, -0.7834, -0.6560,
-0.5287, -0.4013, -0.4013, -0.4013, -0.4013],
[-0.3657, -0.5020, -0.6382, -0.7745, -0.9107, -0.7834, -0.6560,
-0.5287, -0.4013, -0.4013, -0.4013, -0.4013],
[-0.3657, -0.5020, -0.6382, -0.7745, -0.9107, -0.7834, -0.6560,
-0.5287, -0.4013, -0.4013, -0.4013, -0.4013]]]], device='cuda:0')
| torch2trt.converters.upsample.test_bilinear_mode | float32 | [(1, 1, 3, 3)] | {} | 1.45E+00 | 4.01e+04 | 2.21e+04 | 0.0408 | 0.0935 |

[SrivastavaKshitij]

Fix for interpolate bilinear mode problem will come with TRT 7.1

[Jockeypan]

@SrivastavaKshitij tks 👍