How to enable --best option in the trtexec tools when using C++ API

[ianWeichengZhang]

Description

How to enable --best option in the trtexec tools when using C++ API?

Environment

TensorRT Version: 8.2.4.2
NVIDIA GPU: 3080ti
NVIDIA Driver Version:
CUDA Version: 11.6
CUDNN Version: 8.2
Operating System:
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):

Relevant Files

Steps To Reproduce

[ianWeichengZhang]

I tried the code belows to enable int8 precision and enables all precision next.

config->setInt8Calibrator(nullptr);
config->setFlag(nvinfer1::BuilderFlag::kINT8);

However I got error belows.

4: [standardEngineBuilder.cpp::initCalibrationParams::1402] Error Code 4: Internal Error (Calibration failure occurred with no scaling factors detected. This could be due to no int8 calibrator or insufficient custom scales for network layers. Please see int8 sample to setup calibration correctly.)

[zerollzeng]

--best means config->setFlag(nvinfer1::BuilderFlag::kINT8);config->setFlag(nvinfer1::BuilderFlag::kFP16);

[zerollzeng]

If you just want to build an engine that need calibration in INT8, you can try Polygraphy.

[ttyio]

Closing since no activity for more than 3 weeks, pls reopen if you still have question. thanks!