q buffer sequence length not multiple of 16 lead to inference failure

[de1star]

Description

Hi TensorRT team, appreciate your great work! I were trying to increase the inference speed of stable diffusion by adding plugins like fMHA and fMHCA into UNet. The size of input image of my model were 512*768, 640*640 and 768*512, so I set a dynamic shapes when transforming torch model to onnx, and onnx to tensorRT engine. However, when the tensorRT engine was used with an input image which size was 640*640, ValueError appeared:

.
Will you consider to support size like 640*640 for fMHCA?

Environment

TensorRT Version: 8.5.1.7
NVIDIA GPU: A100
NVIDIA Driver Version: 460.32.03
CUDA Version: cuda-11.6
CUDNN Version: 8.7.0.84
Operating System: CentOS
Python Version (if applicable): 3.8
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 1.12.1+cuda113_cudnn8.3.2
Baremetal or Container (if so, version):

Relevant Files

https://github.com/NVIDIA/TensorRT/tree/release/8.5/demo/Diffusion

[zerollzeng]

@nvpohanh Is this a known issue? Thanks!

[nvpohanh]

Let me ask internally about this.

[zerollzeng]

@de1star Can you try our latest code? Maybe this is a fixed issue.

BTW fMHCA and fMHA do not have shape requirements today and we require the input to be a multiple of 8 to correctly calculate the latent image size.

[de1star]

Thanks, I will have a try!

[de1star]

Hi tensorRT team @zerollzeng @nvpohanh , I have used tensorRT-8.5.2.2 and solved the problem. Thanks for your help!

[Jack47]

@de1star so we may close this now?

[de1star]

@de1star so we may close this now?

Yes