tensorRT8.x NCHW format

[alicera]

Description

if using the tensorRT engine, the network input should be NCHW now ?

the cv::Mat &input should be NCHW?

    const cv::Mat &input
    void *data = malloc(Len);
    memcpy(data, input.ptr<float>(0), Len);

    cudaMemcpyAsync(buffers[0], data, \
    inputLen, cudaMemcpyHostToDevice, stream);

    // infernece
    m_context->enqueueV2(buffers, stream, nullptr);

Environment

TensorRT Version: 8.6

[zerollzeng]

It's defined by your model by default, or if you specify it when build the engine, it's the format you specify. A quick start would be trtexec --help, you should see

  --inputIOFormats=spec              Type and format of each of the input tensors (default = all inputs in fp32:chw)
                                     See --outputIOFormats help for the grammar of type and format list.
                                     Note: If this option is specified, please set comma-separated types and formats for all
                                           inputs following the same order as network inputs ID (even if only one input
                                           needs specifying IO format) or set the type and format once for broadcasting.
  --outputIOFormats=spec             Type and format of each of the output tensors (default = all outputs in fp32:chw)
                                     Note: If this option is specified, please set comma-separated types and formats for all
                                           outputs following the same order as network outputs ID (even if only one output
                                           needs specifying IO format) or set the type and format once for broadcasting.
                                     IO Formats: spec  ::= IOfmt[","spec]
                                                 IOfmt ::= type:fmt
                                               type  ::= "fp32"|"fp16"|"int32"|"int8"
                                               fmt   ::= ("chw"|"chw2"|"chw4"|"hwc8"|"chw16"|"chw32"|"dhwc8"|
                                                          "cdhw32"|"hwc"|"dla_linear"|"dla_hwc4")["+"fmt]
...
  --precisionConstraints=spec        Control precision constraint setting. (default = none)
                                       Precision Constraints: spec ::= "none" | "obey" | "prefer"
                                         none = no constraints
                                         prefer = meet precision constraints set by --layerPrecisions/--layerOutputTypes if possible
                                         obey = meet precision constraints set by --layerPrecisions/--layerOutputTypes or fail
                                                otherwise

[zerollzeng]

For more info please refer to https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#reformat-free-network-tensors

[ttyio]

closing since no activity for more than 3 weeks, pls reopen if you still have question, thanks!