Unable to convert HTC with DCN and multi-scale training model to tensor-rt

[Prasoonk02]

Hi,

I'm trying to convert a custom mmdet model that was created based on this pre-trained HTC model available from https://github.com/open-mmlab/mmdetection/tree/master/configs/htc/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco.py

When I try to convert the model, I get the error as "'HybridTaskCascadeRoIHead' object has no attribute 'semantic_head'"

Error log is:
/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/models/builder.py:53: UserWarning: train_cfg and test_cfg is deprecated, please specify them in model
'please specify them in model', UserWarning)
load checkpoint from local path: epoch_74.pth
2021-12-16 16:02:15,813 - root - INFO - DeformConv2dPack backbone.layer2.0.conv2 is upgraded to version 2.
2021-12-16 16:02:15,815 - root - INFO - DeformConv2dPack backbone.layer2.1.conv2 is upgraded to version 2.
2021-12-16 16:02:15,817 - root - INFO - DeformConv2dPack backbone.layer2.2.conv2 is upgraded to version 2.
2021-12-16 16:02:15,818 - root - INFO - DeformConv2dPack backbone.layer2.3.conv2 is upgraded to version 2.
2021-12-16 16:02:15,820 - root - INFO - DeformConv2dPack backbone.layer3.0.conv2 is upgraded to version 2.
2021-12-16 16:02:15,823 - root - INFO - DeformConv2dPack backbone.layer3.1.conv2 is upgraded to version 2.
2021-12-16 16:02:15,825 - root - INFO - DeformConv2dPack backbone.layer3.2.conv2 is upgraded to version 2.
2021-12-16 16:02:15,828 - root - INFO - DeformConv2dPack backbone.layer3.3.conv2 is upgraded to version 2.
2021-12-16 16:02:15,830 - root - INFO - DeformConv2dPack backbone.layer3.4.conv2 is upgraded to version 2.
2021-12-16 16:02:15,833 - root - INFO - DeformConv2dPack backbone.layer3.5.conv2 is upgraded to version 2.
2021-12-16 16:02:15,835 - root - INFO - DeformConv2dPack backbone.layer3.6.conv2 is upgraded to version 2.
2021-12-16 16:02:15,837 - root - INFO - DeformConv2dPack backbone.layer3.7.conv2 is upgraded to version 2.
2021-12-16 16:02:15,840 - root - INFO - DeformConv2dPack backbone.layer3.8.conv2 is upgraded to version 2.
2021-12-16 16:02:15,842 - root - INFO - DeformConv2dPack backbone.layer3.9.conv2 is upgraded to version 2.
2021-12-16 16:02:15,844 - root - INFO - DeformConv2dPack backbone.layer3.10.conv2 is upgraded to version 2.
2021-12-16 16:02:15,847 - root - INFO - DeformConv2dPack backbone.layer3.11.conv2 is upgraded to version 2.
2021-12-16 16:02:15,849 - root - INFO - DeformConv2dPack backbone.layer3.12.conv2 is upgraded to version 2.
2021-12-16 16:02:15,852 - root - INFO - DeformConv2dPack backbone.layer3.13.conv2 is upgraded to version 2.
2021-12-16 16:02:15,854 - root - INFO - DeformConv2dPack backbone.layer3.14.conv2 is upgraded to version 2.
2021-12-16 16:02:15,857 - root - INFO - DeformConv2dPack backbone.layer3.15.conv2 is upgraded to version 2.
2021-12-16 16:02:15,859 - root - INFO - DeformConv2dPack backbone.layer3.16.conv2 is upgraded to version 2.
2021-12-16 16:02:15,861 - root - INFO - DeformConv2dPack backbone.layer3.17.conv2 is upgraded to version 2.
2021-12-16 16:02:15,864 - root - INFO - DeformConv2dPack backbone.layer3.18.conv2 is upgraded to version 2.
2021-12-16 16:02:15,866 - root - INFO - DeformConv2dPack backbone.layer3.19.conv2 is upgraded to version 2.
2021-12-16 16:02:15,868 - root - INFO - DeformConv2dPack backbone.layer3.20.conv2 is upgraded to version 2.
2021-12-16 16:02:15,871 - root - INFO - DeformConv2dPack backbone.layer3.21.conv2 is upgraded to version 2.
2021-12-16 16:02:15,873 - root - INFO - DeformConv2dPack backbone.layer3.22.conv2 is upgraded to version 2.
2021-12-16 16:02:15,876 - root - INFO - DeformConv2dPack backbone.layer4.0.conv2 is upgraded to version 2.
2021-12-16 16:02:15,883 - root - INFO - DeformConv2dPack backbone.layer4.1.conv2 is upgraded to version 2.
2021-12-16 16:02:15,888 - root - INFO - DeformConv2dPack backbone.layer4.2.conv2 is upgraded to version 2.
/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/apis/inference.py:47: UserWarning: Class names are not saved in the checkpoint's meta data, use COCO classes by default.
warnings.warn('Class names are not saved in the checkpoint's '
/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
Traceback (most recent call last):
File "mmtest.py", line 72, in
trt_model = mmdet2trt(config_file, checkpoint_file, opt_shape_param=opt_shape_param, fp16_mode=True, max_workspace_size=max_workspace_size)
File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/mmdet2trt.py", line 110, in mmdet2trt
torch_model, TwoStageDetectorWraper, wrap_config=wrap_config)
File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/builder.py", line 41, in build_wraper
wrap_model = WRAPER_DICT[model_type](module, **kwargs)
File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/detectors/two_stage.py", line 34, in init
mmdet_roi_head, StandardRoIHeadWraper, wrap_config=wrap_config)
File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/builder.py", line 41, in build_wraper
wrap_model = WRAPER_DICT[model_type](module, **kwargs)
File "/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection-to-tensorrt/mmdet2trt/models/roi_heads/htc_roi_head.py", line 20, in init
if module.semantic_head is not None:
File "/home/monster/.conda/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1178, in getattr
type(self).name, name))
AttributeError: 'HybridTaskCascadeRoIHead' object has no attribute 'semantic_head'
(open-mmlab) monster@icvmonster:~/Documents/Wenn/Dd/tensorrt/27_11/mmdetection$

===========================================================================

So, I just tried to convert the HTC model (https://github.com/open-mmlab/mmdetection/tree/master/configs/htc/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco.py ) to tensor-rt, I got the error as "Segmentation Fault (Core Dumped)"
The complete log for this is:
load checkpoint from local path: ckpt/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco_20200312-946fd751.pth
/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/datasets/utils.py:69: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
load checkpoint from local path: ckpt/htc_x101_64x4d_fpn_dconv_c3-c5_mstrain_400_1400_16x1_20e_coco_20200312-946fd751.pth
/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
/home/monster/Documents/Wenn/Dd/tensorrt/27_11/mmdetection/mmdet/core/anchor/anchor_generator.py:370: UserWarning: single_level_grid_anchors would be deprecated soon. Please use single_level_grid_priors
'single_level_grid_anchors would be deprecated soon. '
[12/15/2021-08:45:18] [TRT] [I] [MemUsageChange] Init CUDA: CPU +170, GPU +0, now: CPU 2324, GPU 5821 (MiB)
[12/15/2021-08:45:18] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 2324 MiB, GPU 5821 MiB
[12/15/2021-08:45:18] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 2387 MiB, GPU 5821 MiB
unknown interpolate type, use linear instead.
unknown interpolate type, use linear instead.
unknown interpolate type, use linear instead.
unknown interpolate type, use linear instead.
Warning: Encountered known unsupported method torch.Tensor.new_tensor
Warning: Encountered known unsupported method torch.Tensor.new_tensor
[12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1397) [ElementWise]_output and (Unnamed Layer* 1401) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1406) [ElementWise]_output and (Unnamed Layer* 1410) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1415) [ElementWise]_output and (Unnamed Layer* 1419) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:21] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1424) [ElementWise]_output and (Unnamed Layer* 1428) [Shuffle]_output: first input has type Float but second input has type Int32.
Warning: Encountered known unsupported method torch.Tensor.new_tensor
Warning: Encountered known unsupported method torch.Tensor.new_tensor
[12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1931) [ElementWise]_output and (Unnamed Layer* 1935) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1940) [ElementWise]_output and (Unnamed Layer* 1944) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1949) [ElementWise]_output and (Unnamed Layer* 1953) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:22] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 1958) [ElementWise]_output and (Unnamed Layer* 1962) [Shuffle]_output: first input has type Float but second input has type Int32.
Warning: Encountered known unsupported method torch.Tensor.new_tensor
Warning: Encountered known unsupported method torch.Tensor.new_tensor
[12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 2483) [ElementWise]_output and (Unnamed Layer* 2487) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 2492) [ElementWise]_output and (Unnamed Layer* 2496) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 2501) [ElementWise]_output and (Unnamed Layer* 2505) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:23] [TRT] [W] IElementWiseLayer with inputs (Unnamed Layer* 2510) [ElementWise]_output and (Unnamed Layer* 2514) [Shuffle]_output: first input has type Float but second input has type Int32.
[12/15/2021-08:45:24] [TRT] [W] Half2 support requested on hardware without native FP16 support, performance will be negatively affected.
[12/15/2021-08:45:25] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1
[12/15/2021-08:45:25] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 3221, GPU 2877 (MiB)
[12/15/2021-08:45:25] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +110, GPU +44, now: CPU 3331, GPU 2921 (MiB)
[12/15/2021-08:45:25] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[12/15/2021-08:45:32] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[12/15/2021-08:46:39] [TRT] [I] Detected 1 inputs and 4 output network tensors.
[12/15/2021-08:46:40] [TRT] [I] Total Host Persistent Memory: 593904
[12/15/2021-08:46:40] [TRT] [I] Total Device Persistent Memory: 581303296
[12/15/2021-08:46:40] [TRT] [I] Total Scratch Memory: 520224768
[12/15/2021-08:46:40] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 340 MiB, GPU 1236 MiB
[12/15/2021-08:46:41] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 1403.19ms to assign 51 blocks to 470 nodes requiring 1088816655 bytes.
[12/15/2021-08:46:41] [TRT] [I] Total Activation Memory: 1088816655
[12/15/2021-08:46:41] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1
[12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 3646, GPU 3759 (MiB)
[12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +10, now: CPU 3646, GPU 3769 (MiB)
[12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +290, GPU +731, now: CPU 290, GPU 731 (MiB)
[12/15/2021-08:46:41] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1
[12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +1, GPU +8, now: CPU 3645, GPU 3745 (MiB)
[12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 3645, GPU 3753 (MiB)
[12/15/2021-08:46:41] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +1592, now: CPU 291, GPU 2323 (MiB)
[12/15/2021-08:46:57] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1
[12/15/2021-08:46:57] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1
WARNING:root:Can not load dataset from config. Use default CLASSES instead.
Segmentation fault (core dumped)

======================================================
Can you please help me with this?

[grimoire]

Please repo you enviroment. You can use this script python tools/collect_env.py