Converting DETR models run into Errors when using --layer-precisions

[Faisal-Hajari]

Description

My work is to turn DETR into tensorRT, when using the flag --fp16 it works. however, my plan is to turn the backbone layers to fp16 and keep the transformer layers in fp32. When I do so I get the following error message.

the same code can turn the backbone to fp32 and the transformer into fp16.
it seems the issues that there's foreign kernels most likely coming from Myelin. how to fix this ? and if I can't should I roll back tensorrt before Myelin was supported ?

Environment

TensorRT Version: I'm using Polygraphy Version: 0.49.0

NVIDIA GPU: A6000

NVIDIA Driver Version: 525.116.03

CUDA Version: 12.0

Operating System:

Python Version (if applicable): 3.10

Relevant Files

Model link:

Steps To Reproduce

"polygraphy convert detr.onnx -o detr_mixed.engine --fp16 --precision-constraints obey --layer-precisions /backbone/*:fp16 else:fp32" for the layers in layer-precisions I used a python script to extract all names of onnx layers

[zerollzeng]

Which TRT version you are using? Could you please try latest TRT 9.2? If it still doesn't work please share the onnx model here and we will take a look.

@pranavm-nvidia Do we support syntax like --layer-precisions /backbone/*:fp16 else:fp32 in polygraphy?

[Faisal-Hajari]

so for the support of regex with polygraphy. it doesn't work. because the code throws a * not found in layers { _list the layers of the model_}.

I think the version is 8.6 (based on what's installed in pip).

[pranavm-nvidia]

@zerollzeng Not currently. It is possible to write a script with a custom postprocessing function (where you can have arbitrarily complex logic for marking layer precisions) which you can pass to polygraphy run/polygraphy convert. See this example for details.

[ttyio]

closing since this is already solved, thanks all!