-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TensorRT 8 Support #55
Comments
TensorRT 8 changed the way plugins work. I did not have time to look into it yet. Contributions are very welcomed! |
Usually the computational part of the plugin stays the same and only the interface around it changes, so it should not be too big of a change. |
I am also facing same issue, could you please help me. I used tensorrt:21.08 got this error /yolov4-triton-tensorrt/layers/yololayer.h(59): error: member function declared with "override" does not override a base class member /yolov4-triton-tensorrt/layers/yololayer.h(108): error: function "nvinfer1::IPluginV2IOExt::configurePlugin(const nvinfer1::Dims *, int32_t, const nvinfer1::Dims *, int32_t, const nvinfer1::DataType *, const nvinfer1::DataType *, const __nv_bool *, const __nv_bool *, nvinfer1::PluginFormat, int32_t)" /yolov4-triton-tensorrt/layers/yololayer.h(59): warning: function "nvinfer1::IPluginV2::enqueue(int32_t, const void *const *, void *const *, void *, cudaStream_t)" is hidden by "nvinfer1::YoloLayerPlugin::enqueue" -- virtual function override intended? /yolov4-triton-tensorrt/layers/yololayer.h(89): warning: function "nvinfer1::IPluginV2Ext::configurePlugin(const nvinfer1::Dims *, int32_t, const nvinfer1::Dims *, int32_t, const nvinfer1::DataType *, const nvinfer1::DataType *, const __nv_bool *, const __nv_bool *, nvinfer1::PluginFormat, int32_t)" is hidden by "nvinfer1::YoloLayerPlugin::configurePlugin" -- virtual function override intended? /yolov4-triton-tensorrt/layers/yololayer.h(91): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::detachFromContext" is incompatible with that of overridden function "nvinfer1::IPluginV2Ext::detachFromContext" /yolov4-triton-tensorrt/layers/yololayer.h(89): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::configurePlugin" is incompatible with that of overridden function "nvinfer1::IPluginV2IOExt::configurePlugin(const nvinfer1::PluginTensorDesc *, int32_t, const nvinfer1::PluginTensorDesc *, int32_t)" /yolov4-triton-tensorrt/layers/yololayer.h(87): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::attachToContext" is incompatible with that of overridden function "nvinfer1::IPluginV2Ext::attachToContext" /yolov4-triton-tensorrt/layers/yololayer.h(85): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::canBroadcastInputAcrossBatch" is incompatible with that of overridden function "nvinfer1::IPluginV2Ext::canBroadcastInputAcrossBatch" /yolov4-triton-tensorrt/layers/yololayer.h(83): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::isOutputBroadcastAcrossBatch" is incompatible with that of overridden function "nvinfer1::IPluginV2Ext::isOutputBroadcastAcrossBatch" /yolov4-triton-tensorrt/layers/yololayer.h(81): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getOutputDataType" is incompatible with that of overridden function "nvinfer1::IPluginV2Ext::getOutputDataType" /yolov4-triton-tensorrt/layers/yololayer.h(79): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getPluginNamespace" is incompatible with that of overridden function "nvinfer1::IPluginV2::getPluginNamespace" /yolov4-triton-tensorrt/layers/yololayer.h(77): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::setPluginNamespace" is incompatible with that of overridden function "nvinfer1::IPluginV2::setPluginNamespace" /yolov4-triton-tensorrt/layers/yololayer.h(75): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::clone" is incompatible with that of overridden function "nvinfer1::IPluginV2Ext::clone" /yolov4-triton-tensorrt/layers/yololayer.h(75): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::clone" is incompatible with that of overridden function "nvinfer1::IPluginV2::clone" /yolov4-triton-tensorrt/layers/yololayer.h(73): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::destroy" is incompatible with that of overridden function "nvinfer1::IPluginV2::destroy" /yolov4-triton-tensorrt/layers/yololayer.h(71): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getPluginVersion" is incompatible with that of overridden function "nvinfer1::IPluginV2::getPluginVersion" /yolov4-triton-tensorrt/layers/yololayer.h(69): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getPluginType" is incompatible with that of overridden function "nvinfer1::IPluginV2::getPluginType" /yolov4-triton-tensorrt/layers/yololayer.h(65): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::supportsFormatCombination" is incompatible with that of overridden function "nvinfer1::IPluginV2IOExt::supportsFormatCombination" /yolov4-triton-tensorrt/layers/yololayer.h(63): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::serialize" is incompatible with that of overridden function "nvinfer1::IPluginV2::serialize" /yolov4-triton-tensorrt/layers/yololayer.h(61): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getSerializationSize" is incompatible with that of overridden function "nvinfer1::IPluginV2::getSerializationSize" /yolov4-triton-tensorrt/layers/yololayer.h(57): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getWorkspaceSize" is incompatible with that of overridden function "nvinfer1::IPluginV2::getWorkspaceSize" /yolov4-triton-tensorrt/layers/yololayer.h(55): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::terminate" is incompatible with that of overridden function "nvinfer1::IPluginV2::terminate" /yolov4-triton-tensorrt/layers/yololayer.h(53): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::initialize" is incompatible with that of overridden function "nvinfer1::IPluginV2::initialize" /yolov4-triton-tensorrt/layers/yololayer.h(51): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getOutputDimensions" is incompatible with that of overridden function "nvinfer1::IPluginV2::getOutputDimensions" /yolov4-triton-tensorrt/layers/yololayer.h(46): error: exception specification for virtual function "nvinfer1::YoloLayerPlugin::getNbOutputs" is incompatible with that of overridden function "nvinfer1::IPluginV2::getNbOutputs" /yolov4-triton-tensorrt/layers/yololayer.h(133): error: exception specification for virtual function "nvinfer1::YoloPluginCreator::getPluginNamespace" is incompatible with that of overridden function "nvinfer1::IPluginCreator::getPluginNamespace" /yolov4-triton-tensorrt/layers/yololayer.h(128): error: exception specification for virtual function "nvinfer1::YoloPluginCreator::setPluginNamespace" is incompatible with that of overridden function "nvinfer1::IPluginCreator::setPluginNamespace" /yolov4-triton-tensorrt/layers/yololayer.h(126): error: exception specification for virtual function "nvinfer1::YoloPluginCreator::deserializePlugin" is incompatible with that of overridden function "nvinfer1::IPluginCreator::deserializePlugin" /yolov4-triton-tensorrt/layers/yololayer.h(124): error: exception specification for virtual function "nvinfer1::YoloPluginCreator::createPlugin" is incompatible with that of overridden function "nvinfer1::IPluginCreator::createPlugin" /yolov4-triton-tensorrt/layers/yololayer.h(122): error: exception specification for virtual function "nvinfer1::YoloPluginCreator::getFieldNames" is incompatible with that of overridden function "nvinfer1::IPluginCreator::getFieldNames" /yolov4-triton-tensorrt/layers/yololayer.h(120): error: exception specification for virtual function "nvinfer1::YoloPluginCreator::getPluginVersion" is incompatible with that of overridden function "nvinfer1::IPluginCreator::getPluginVersion" /yolov4-triton-tensorrt/layers/yololayer.h(118): error: exception specification for virtual function "nvinfer1::YoloPluginCreator::getPluginName" is incompatible with that of overridden function "nvinfer1::IPluginCreator::getPluginName" /yolov4-triton-tensorrt/layers/yololayer.cu(173): error: object of abstract class type "nvinfer1::YoloLayerPlugin" is not allowed: /yolov4-triton-tensorrt/layers/yololayer.cu(412): error: object of abstract class type "nvinfer1::YoloLayerPlugin" is not allowed: /yolov4-triton-tensorrt/layers/yololayer.cu(419): error: object of abstract class type "nvinfer1::YoloLayerPlugin" is not allowed: 33 errors detected in the compilation of "/yolov4-triton-tensorrt/layers/yololayer.cu". make[2]: *** [CMakeFiles/layerplugin.dir/build.make:65: CMakeFiles/layerplugin.dir/layers/layerplugin_generated_yololayer.cu.o] Error 1 |
yeah ran into the same issue when I update to jetson jetpack 4.6 from 4.5 and tensorRT upgraded to 8.0.1 |
I followed this pr's changes and got it to at least build now wang-xinyu/tensorrtx#540 Also note need to update the docker image to a TRT8 one, such as |
Thanks a lot.
I will go through it.
…On Wed, Nov 3, 2021 at 9:00 PM chull434 ***@***.***> wrote:
I followed this pr's changes and got it to at least build now
wang-xinyu/tensorrtx#540
<wang-xinyu/tensorrtx#540>
Also note need to update the docker image to a TRT8 one, such as
nvcr.io/nvidia/tensorrt:21.09-py3
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#55 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AWDAXTIJB4DDQHIBALD76KLUKFIRHANCNFSM5F5DGZ3Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
Thanks and Regards
Dr. Revanasiddappa M Biradar
Data Scientist
Bengaluru
|
@Michelvl92 did you find a way to support TRT8? |
I've also ended up here with the same errors trying to use the 21.08 version. Curious about the status? |
No support yet. We will update this to TensorRT 8 at some point, no roadmap though. PR from someone else will probably be faster. Should only be a matter of applying necessary changes from the patch notes from nvidia to the new plugin interface. |
Newest v1.4.0 release comes with support for TRT 8. Tested with NGC TensorRT 21.10 and Triton 21.10 with 470 driver. Performance gain compared to previous 1.3.0 release is at about 19.5%. Did not have the time to run and check the python client, so feel free to report any issues with that. |
I have tried to build yolov4 with the tutorial, but instead of using Nvidia tensorrt docker version 21.03 version, I used 21.09 (with TensorRT 8.0.3) which resulted in the following errors:
The text was updated successfully, but these errors were encountered: