[Question]Doesn't torch tensorrt support LSTM-based decoder optimization??

[koliaok]

❓ Question

Doesn't torch tensorrt support LSTM-based decoder optimization? The reason for asking this question is that the model forward and model test structures learned in the seq2seq structure are different (beam search, sequence inference ..), and the optimized model cannot be used by inputting only training forward logic.

Environment

Tensorrt 22.03 docker image:
https://docs.nvidia.com/deeplearning/tensorrt/container-release-notes/rel_22-03.html#rel_22-03

[github-actions]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

[ncomly-nvidia]

Hi @koliaok can you please elaborate? Are you asking if Torch-TRT supports seq2seq models? Any specific models you're looking at?

the optimized model cannot be used by inputting only training forward logic.
Can you please explain further? Torch-TRT is an inference only compiler.

[github-actions]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days