Stable Diffusion BUG

[triple-Mu]

Description

When I execute LD_PRELOAD=/workspace/project/TensorRT/build/out/libnvinfer_plugin.so python3 demo-diffusion.py "a beautiful photograph of Mt. Fuji during cherry blossom" --hf-token=$HF_TOKEN -v, I am not able to successfully run the given example.

[E] 9: [pluginV2Builder.cpp::reportPluginError::23] Error Code 9: Internal Error (fMHA_0: could not find any supported formats consistent with input/output data types)
[E] 2: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[!] Invalid Engine. Please ensure the engine was built correctly

https://github.com/NVIDIA/TensorRT/tree/release/8.5/demo/Diffusion

Environment

TensorRT Version: 8.5.2.2
NVIDIA GPU: v100
NVIDIA Driver Version:470.82.01
CUDA Version: 11.4
CUDNN Version: 8.4
Operating System: ubuntu
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):

Relevant Files

Steps To Reproduce

[nvpohanh]

The current demoDiffusion code is only supported on Turing+ GPUs and is not supported on Volta GPUs. You can try adding --minimal-optimizations to disable the plugins and see if it works.

[triple-Mu]

I test in A100-40G. It work well!

[awsankur]

FYI. I tested with --onnx-minimal-optimization flag on a V100 GPU but got the same error as above.

[nekorobov]

Hi, just checking -- have you cleared onnx and engine dirs before calling it with --onnx-minimal-optimization? It is required to do because the script will not regenerate onnx graphs if the onnx file already exists in the folder unless --force-onnx-optimize is specified.

[lizhengwei1992]

got the error on V100 ..
(fMHA_0: could not find any supported formats consistent with input/output data types)

I follow the readme of deme/Diffusion. release : 8.5.3
env:
TensorRT Version: 8.5.3.1
NVIDIA GPU: v100
NVIDIA Driver Version: 520.61.05
CUDA Version: 11.8
CUDNN Version: 8.6
Operating System: ubuntu