Plugin If the output's size is calculated from "enqueue", how can I determine the Dims in "getOutputDimensions"?

[jianxiezhendong]

Now I write a op with unknow output size which is calculated from the "enqueue" function. But fisrtly, the function "getOutputDimensions" needs a certain size. Could you give me some idea? Thank you!

[ttyio]

Hello @jianxiezhendong , thanks for reporting.
Does your plugin has data dependent dimensions (e.g, NoneZeros in ONNX), we do not support this yet.
Otherwise you can build your expression in your plugin:

simple sample:
https://github.com/NVIDIA/TensorRT/blob/release/7.2/plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp#L159
advanced sample:
https://github.com/NVIDIA/TensorRT/blob/release/7.2/plugin/embLayerNormPlugin/embLayerNormPlugin.cpp#L188

[rajeevsrao]

@jianxiezhendong if implementing a plugin on IPluginV2DynamicExt interface, the getOutputDimensions method should return a dimension expression for the corresponding input dim range to the plugin layer, and this query is used by TensorRT builder. The runtime value (during enqueue) is based on the input sizes fed during enqueue. There is no user query available for this at the moment.

[ttyio]

Closing since no activity in this thread for more than 3 weeks, please reopen if you still have question, thanks!

[Tengxu-Sun]

Hello @jianxiezhendong , thanks for reporting. Does your plugin has data dependent dimensions (e.g, NoneZeros in ONNX), we do not support this yet. Otherwise you can build your expression in your plugin:

simple sample: https://github.com/NVIDIA/TensorRT/blob/release/7.2/plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp#L159 advanced sample: https://github.com/NVIDIA/TensorRT/blob/release/7.2/plugin/embLayerNormPlugin/embLayerNormPlugin.cpp#L188

hi @ttyio. Is there any way to implement a NoneZeros plugin since output's size is not sure.

[rajeevsrao]

@Tengxu-Sun with the current TensorRT plugin interface (plugin outputs are an expression of input shapes), there is no way to implement NonZero. We have plans to address this in a future release.

[yuefanhao]

@rajeevsrao @ttyio Hi, Is there any way to implement nonzero and unique through the tensorrt plugin? which have no explicit expression between the output dimension and the input dimension.