Invalid workspace buffer in custom plugin enqueue

[Jsy0220]

Hi, I have implemented a custom plugin inherited from nvinfer1::IPluginV2DynamicExt and this layer need some runtime cuda memory, so getWorkspaceSize returns non-zero. But when calling enquue, the pointer to workspace is still nullptr. Is there anything wrong ? Following are the example codes:

TensorRT 8.4.0.6

[Jsy0220]

It is wired but getWorkspaceSize seems never be called. So how can i request the extra cuda memory used in enqeue?

[zerollzeng]

I'm checking this with plugin expert, will come back to you when I have an answer :)

[zerollzeng]

I did an experiment on my plugin code, I can't reproduce this error. @Jsy0220 does your getWorkspaceSize function signature end with noexcept?

More info about the workspace:
A layer's workspace memory is not managed by the user. It is managed by TensorRT and reused across layers. getWorkspaceSize tells TensorRT how much workspace memory a layer needs during the engine building stage (which can be zero). TensorRT allocates the maximum size of workspace memory among all layers during the inference stage. void* workspace is the allocated workspace memory that is passed by TensorRT to a layer.

[Jsy0220]

Yes, it ends with noexcept

Thank you for your answer. I have known the reason.

[zerollzeng]

May I know what's the reason?

[demuxin]

After modifying the getWorkspaceSize function, if not rebuild the model engine, the workspace is still nullptr.
@zerollzeng