[Feature]: Support exporting to ONNX with batch and dynamic shape #5039

RunningLeon · 2021-04-21T11:56:25Z

This PR includes:

Support exporting to ONNX with batch and dynamic shape
Support evaluate ONNX model with ONNXRuntime backend.
Update related documents

…mmlab#4781) * fix fcos deploy bug and add fcos fsaf ssd head unittest case * add pickle file to be used in test case * cancel deploy_nms_pre and add with_nms * optimized implementation * optimaze code * fix docstring typo

* add area name grouped map bar plots * update function name, add comments, fix typo * make all changes optional, add adjustable area region support, add area stat plots * clean unnecessary comma

* Add mmdet_handler * Add mmdet2torchserve * Move serve to docker folder

* Update doc * Update docs * Update docs * Update 1_exist_data_model.md Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>

…lab#4796) * Support batch infer in RetinaNet * Suport batch multiclass_nms * Revert multiclass_nms * Fix api deprecated warning * do not repeat anchors * Move img_shapes * Update Yolov3 * Support FCOS * Support RPN * Fix RPN topk_inds error * make batch exportable to onnx for yolohead * make fcos_head exportable to onnx with batch dim * Support ATSS * Support CornerNet and centripetalNet * Update RetinaNet and delta_xywh * Remove ugly code * Remove ugly code of FCOS * Remove ugly code of ATSS/YOLOV3 * Support RPN and revert bbox_head * expand anchors to batch and remove BG class when use deploy_nms_pre * Update * Use dim=-1 instead of dim=2 * Rename anchor_head method * Keep the original format output when nms is not use * Rename method and unified code style * Fix paa_head and unittest * Fix FASF onnx export error * Fix error * fix single stage img_shapes for onnx * move conf_thr * fix rpn_head for onnx * Add distance2bbox unittest * Remove TODO * Fix RPN * Update docstrs * Faster R-CNN support batch_infer * update for exporting to onnx * add dummy onnx::nms for anchor head * add dummy onnx::nms for rpn_head * update bbox head for fasterrcnn batch export to onnx * update to support pytorch2onnx with batch * add onnx helper * add dummn onnx:nms for yolo head * add dummy onnx:nms for fcos head * add topk to rpn head for onnx * update pytorch2onnx doc * ignore background class for anchor head if not use_sigmoid_cls * change to 1d indexing for onnx_helper * Support faster rcnn * Support faster rcnn * Fix Lint * change max_output_boxes_per_class defaults to 200 * set value to zero for bboxes in rpn head as well * set deploy_nms_pre in rpn head * add topk of deploy_nms_pre in bbox head before nms * Add docstr * add deploy_nms_pre for rpn head and remove topk after nms * Fix docstr * Update docstr * add with_nms control over onnx exporting * update doc for pytorch2onnx * support trt backend for exporting to onnx * update onnx2tensorrt and its doc * Update code * Update docstr * MaskRCNN support batch infer * Update * change dtype in range in anchor_free_head * change to support onnx2tensorrt with batch * update coder for onnx * move dynamic clip to onnx_helper * move dynamic clip to onnx_helper * sort trt_dets in onnx2tensorrt * reorganize output structure to support batch export to onnx * support maskrcnn to onnx with TensorRT backend * fix after nms topk for batch export * Revert "support maskrcnn to onnx with TensorRT backend" This reverts commit 37fcefb. * Revert "Update" from haian for mask export This reverts commit ce725bf. * remove import torch from simple_test in rpn * resolve comment for code style * Revert "MaskRCNN support batch infer" This reverts commit aa6ced5. * fix code style Co-authored-by: hhaAndroid <1286304229@qq.com>

* Fix some typos Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com>

…mlab#4880) Co-authored-by: Yuxin Wu <ppwwyyxx@users.noreply.github.com>

* Update get_started.md * Update get_started.md

* add change log for v2.11.0 * fix typo * version 2.11.0 * fix typo * to meet the specification of change log * Update changelog.md Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>

* Support faster rcnn * Fix Lint * Add docstr * Fix docstr * Update docstr * Update code * Update docstr * MaskRCNN support batch infer * Update * Delete todo * Fix error * Fix comments

* pytorch2onnx and onnx2trt doc * fix onnx2tensorrt.md * update md * fix error link in onnx2tensorrt.md * fix typo in md * updata docs * updata docs * update docs * update docs * fix the bug in docs * remove max_shape in onnx2tensorrt.md

* Support Concatenate dataset for testing * Support faster rcnn * Fix Lint * Add docstr * Fix docstr * Update docstr * Allow onnx mrcnn * Update code * Update docstr * MaskRCNN support batch infer * Update * Add mrcnn trt export * [Refactor]: Add deprecate warnings of LoadImage (open-mmlab#4813) * Remove LoadImage * Add deprecation warning * [feature] improve coco_error_analysis.py (open-mmlab#4712) * add area name grouped map bar plots * update function name, add comments, fix typo * make all changes optional, add adjustable area region support, add area stat plots * clean unnecessary comma * [Fix]: minor syntax typo fix (open-mmlab#4836) * [Feature]: Add mmdet2torchserve tool (open-mmlab#4810) * Add mmdet_handler * Add mmdet2torchserve * Move serve to docker folder * Support no gt test mode (open-mmlab#4827) * Update doc * Update docs * Update docs * Update 1_exist_data_model.md Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> * [Feature]: Add Localization Distillation for Object Detection (open-mmlab#4758) * add Localization Distillation for Object Detection https://arxiv.org/abs/2102.12252 * fix lint * fix lint * fix lint * fix lint * fix lint * fix config * add kd detector * edit loss name * overide setattr but failed * move init_detector into init * seperate ld and gfocal, fix unused param error * small fix * small fix * add test, reload magic function, create kd_loss.py * del teacher model warnings * fix reference * add ignored bbox test fix docstring * small fix * docstring fix * change names * fix * fix * fix test * fix import * fix * docstring fix * fix * retest * add test * docstring fix * Fix some typos (open-mmlab#4858) * Fix some typos Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com> * MaskRCNN support batch infer * Update * cherry pick commits to support maskrcnn with tensorrt * Fix supplement size in simple_test_bboxes (open-mmlab#4860) * Delete todo * Fix error * Use a unified add_dummy_nms_for_onnx for bbox_head * Fix comments * fix lint * remove inplace operation and update doc Co-authored-by: hhaAndroid <1286304229@qq.com> Co-authored-by: David de la Iglesia Castro <daviddelaiglesiacastro@gmail.com> Co-authored-by: fcakyon <34196005+fcakyon@users.noreply.github.com> Co-authored-by: tczhao <evantczhao@gmail.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> Co-authored-by: HikariTJU <30501451+HikariTJU@users.noreply.github.com> Co-authored-by: lizz <innerlee@users.noreply.github.com> Co-authored-by: maningsheng <maningsheng@sensetime.com> Co-authored-by: Ye Liu <yeliudev@outlook.com>

) * fix decoder clamp error on twostage model * update docstring, add more comment

* fix onnxsim to official version * update code

…tch inference (open-mmlab#4940) * add test dynamic and batch * update comment * update doc * update doc

CLAassistant · 2021-04-21T11:56:36Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
14 out of 15 committers have signed the CLA.

✅ fcakyon
✅ tczhao
✅ daavoo
✅ hhaAndroid
✅ QingChuanWS
✅ ppwwyyxx
✅ RunningLeon
✅ yeliudev
✅ innerlee
✅ ZwwWayne
✅ grimoire
✅ Johnson-Wang
✅ jshilong
✅ ZephyrusKylin
❌ hcy226
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

* fix mask output without threshold * mask output compatible with ort and trt * add ort test tool * update parameters of nms for bboxhead and rpnhead * update doc * reorganize to mmdet.core.export * update doc * reorganize import for export * rename ort_test.py * update performance of maskrcnn * fix yolo and onnx2trt * update code style

codecov · 2021-04-22T06:36:19Z

Codecov Report

Merging #5039 (866f20c) into master (2313bd7) will decrease coverage by 0.49%.
The diff coverage is 18.93%.

@@            Coverage Diff             @@
##           master    #5039      +/-   ##
==========================================
- Coverage   66.66%   66.16%   -0.50%     
==========================================
  Files         259      261       +2     
  Lines       20269    20524     +255     
  Branches     3447     3479      +32     
==========================================
+ Hits        13512    13580      +68     
- Misses       6029     6209     +180     
- Partials      728      735       +7

Flag	Coverage Δ
unittests	`66.12% <18.93%> (-0.51%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet/core/__init__.py	`100.00% <ø> (ø)`
mmdet/core/bbox/coder/tblr_bbox_coder.py	`59.21% <0.00%> (-4.17%)`	⬇️
mmdet/core/export/model_wrappers.py	`0.00% <0.00%> (ø)`
mmdet/models/roi_heads/bbox_heads/bbox_head.py	`78.06% <0.00%> (-9.87%)`	⬇️
mmdet/models/roi_heads/standard_roi_head.py	`64.44% <0.00%> (ø)`
mmdet/models/roi_heads/test_mixins.py	`55.08% <0.00%> (-2.78%)`	⬇️
mmdet/models/dense_heads/yolo_head.py	`51.59% <7.14%> (-10.36%)`	⬇️
mmdet/models/dense_heads/fcos_head.py	`76.08% <13.63%> (+5.99%)`	⬆️
mmdet/models/detectors/rpn.py	`62.50% <16.66%> (-4.75%)`	⬇️
mmdet/models/roi_heads/mask_heads/fcn_mask_head.py	`69.41% <21.21%> (-2.37%)`	⬇️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2313bd7...866f20c. Read the comment docs.

ZwwWayne · 2021-04-25T09:23:22Z

Can be merged if the GPU tests are passed. @hhaAndroid

QingChuanWS and others added 22 commits March 25, 2021 10:45

[feature] improve coco_error_analysis.py (open-mmlab#4712)

09923a8

* add area name grouped map bar plots * update function name, add comments, fix typo * make all changes optional, add adjustable area region support, add area stat plots * clean unnecessary comma

[Fix]: minor syntax typo fix (open-mmlab#4836)

864b5e4

[Feature]: Add mmdet2torchserve tool (open-mmlab#4810)

290ae7a

* Add mmdet_handler * Add mmdet2torchserve * Move serve to docker folder

Support no gt test mode (open-mmlab#4827)

025ec3b

* Update doc * Update docs * Update docs * Update 1_exist_data_model.md Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>

Merge branch 'onnx' of github.com:open-mmlab/mmdetection into onnx

640b726

Fix some typos (open-mmlab#4858)

7b671d2

* Fix some typos Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com> * More Signed-off-by: lizz <lizz@sensetime.com>

Fix supplement size in simple_test_bboxes (open-mmlab#4860)

9ef3509

[Refactor]: move pycocotools version check to when it is used (open-m…

304e3db

…mlab#4880) Co-authored-by: Yuxin Wu <ppwwyyxx@users.noreply.github.com>

Use pip install to install mmdetection (open-mmlab#4879)

305ab94

* Update get_started.md * Update get_started.md

Bump version to v2.11.0 (open-mmlab#4870)

fe42371

* add change log for v2.11.0 * fix typo * version 2.11.0 * fix typo * to meet the specification of change log * Update changelog.md Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>

fix lint error (open-mmlab#4885)

36e3e37

[Enhancement]: fix docstr of anchor head (open-mmlab#4883)

0f3e452

[Fix]: Fix bug in mean_ap by 11 points (open-mmlab#4875)

c23ff6d

Support ONNX exportable batch inference of Mask R-CNN (open-mmlab#4871)

26e0a0f

* Support faster rcnn * Fix Lint * Add docstr * Fix docstr * Update docstr * Update code * Update docstr * MaskRCNN support batch infer * Update * Delete todo * Fix error * Fix comments

[fix]: Fix dynamic clip error in box coder for TensorRT (open-mmlab#4905

03adf6b

) * fix decoder clamp error on twostage model * update docstring, add more comment

Update onnx-simplify tool to 3rd party package (open-mmlab#4913)

5e5cbec

* fix onnxsim to official version * update code

[Enhancement]: update pytorch2onnx to test dynamic shape and multi-ba…

f203fe2

…tch inference (open-mmlab#4940) * add test dynamic and batch * update comment * update doc * update doc

Merge branch 'master' of github.com:open-mmlab/mmdetection into onnx

2894dc3

RunningLeon requested a review from ZwwWayne April 21, 2021 11:56

RunningLeon self-assigned this Apr 21, 2021

RunningLeon added the ONNX label Apr 21, 2021

RunningLeon and others added 3 commits April 22, 2021 11:47

Merge branch 'master' of github.com:open-mmlab/mmdetection into onnx

2b13f88

Merge branch 'onnx' of github.com:open-mmlab/mmdetection into onnx

a2ffd3d

zhangshilong and others added 4 commits April 22, 2021 19:32

fix unitest

d49cf27

move CUDAExecutionProvider to first place

3a3c48c

set num_fake_det for dummy nms to speed up exporting to ONNX

2df0fcf

update pytorch2onnx doc

d388aef

ZwwWayne approved these changes Apr 25, 2021

View reviewed changes

suppress import onnxruntime for pytorch test and train

866f20c

ZwwWayne merged commit a4a0a47 into open-mmlab:master Apr 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature]: Support exporting to ONNX with batch and dynamic shape #5039

[Feature]: Support exporting to ONNX with batch and dynamic shape #5039

RunningLeon commented Apr 21, 2021 •

edited

CLAassistant commented Apr 21, 2021 •

edited

codecov bot commented Apr 22, 2021 •

edited

ZwwWayne commented Apr 25, 2021

[Feature]: Support exporting to ONNX with batch and dynamic shape #5039

[Feature]: Support exporting to ONNX with batch and dynamic shape #5039

Conversation

RunningLeon commented Apr 21, 2021 • edited

CLAassistant commented Apr 21, 2021 • edited

codecov bot commented Apr 22, 2021 • edited

Codecov Report

ZwwWayne commented Apr 25, 2021

RunningLeon commented Apr 21, 2021 •

edited

CLAassistant commented Apr 21, 2021 •

edited

codecov bot commented Apr 22, 2021 •

edited