Support Tuple Inputs

[Michael-Equi]

Hello. I am working with a model that takes in a tuple of inputs of different sizes. Is there a way to handle this within the existing TRTorch framework? I am attempting to compile the model with the following settings but get the accompanied error. I am running form source version on commit 247c748.

Compile settings:

compile_settings = {
    "input_shapes": [
        {
            "min": ([1, 3, 180, 320], [1,49,2]),
            "opt": ([1, 3, 180, 320], [1,49,2]),
            "max": ([1, 3, 180, 320], [1,49,2])
        }, # For static size [1, 3, 224, 224]
    ],
#     "op_precision": torch.half # Run with FP16
}

Error:

TypeError: (): incompatible function arguments. The following argument types are supported:
    1. (self: trtorch._C.InputRange, arg0: List[int]) -> None

Invoked with: <trtorch._C.InputRange object at 0x7fda486f67b0>, ([1, 3, 180, 320], [1, 49, 2])

The forward function of my model is the following:

    def forward(self,x,seq):
        
        x = self.input_conv(x)
        x = self.input_fc(x).unsqueeze(0)
        
        actions = self.action_fc(seq)
        x, hidden = self.lstm(actions, (x, x))
        
        output = self.output_fc(x)
        
        return output

Thanks

[narendasan]

We do support multiple inputs. It seems like your syntax is a bit off. Seems like you are using constant sized inputs so these settings should work

compile_settings = {
     "input_shapes" = [[1, 3, 180, 320], [1,49,2]]
}

If you want to do dynamic shape you would do something like this:

compile_settings = {
     "input_shapes" = [{
            "min": (1, 3, 100, 320),
            "opt": (1, 3, 180, 320),
            "max": (1, 3, 260, 320),
        }, {
            "min": [1,10,2],
            "opt": [1,49,2],
            "max": [1,60,2],
        }],
}

Its a bit verbose, but more consistent with how it is done in C++ which at least for now makes it a bit easier to work on. I could see us move to something more similar to what you did as the API stabilizes.

Looking at the forward function of your model, we don't have support for LSTMs yet. We have someone working on it, but I don't think it is going to land soon as its part of a greater RNN and scripting push we are making. You can try to implement a converter yourself for it that works in the context of your application or you can decompose your model around the lstm layers so that you compile the encoder and decoder but leave the LSTM in TorchScript.

[github-actions]

This issue has not seen activity for 30 days, Remove stale label or comment or this will be closed in 5 days