Reformat DETR transformer. #3866
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## detr #3866 +/- ##
==========================================
+ Coverage 61.70% 62.54% +0.84%
==========================================
Files 216 221 +5
Lines 15602 16560 +958
Branches 2646 2788 +142
==========================================
+ Hits 9627 10358 +731
- Misses 5497 5702 +205
- Partials 478 500 +22
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
ZwwWayne
approved these changes
Oct 9, 2020
ZwwWayne
added a commit
that referenced
this pull request
Nov 29, 2020
* supports for DETR transformer * unit test with small data to avoid out of memory in CI * use batch size 1 for unit test to avoid out of memory * move transformer into utils folder and use more small data for unit test * reformat docstring * add more detailed docstring * reforamt * reformat and add build_transformer (#3866) * Supports for DETR position embedding (#3850) * supports for DETR position embedding * reformat docstring * reformat * move transformer_head folder into utils * remove unnecessary comments * reformat * reformat * reformat docstring * reformat * add optional for default parameters * add comments for unit test * avoid inplace in MultiheadAttention and add optional add_residual for FFN * Supports for DETR inference (#3941) * fix forward in FFN * supports detr inference mode with batchsize 1 * add docstring for DETR in detr.py * reformat and add doctest for TransformerHead * reformat, and move the mask construction from simple_test in detr.py to the forward process of transformer_head * rename detr_r50_1x_coco as detr_r50_8x4_1x_coco in config * reformat * mask input_img_shape in img_meta as a general modification of base_detector forward_test * reformat * Supports for DETR hungarian matcher. (#3929) * Add doc of modify loss (#3777) * add doc of modify loss * minor fix * change name * improve the function of simple_test_bboxes (#3853) * update * fix lint error * add bbox_pred_split in sabl_head * fix lint error * fix yapf * fix build error * fix yapf * Clean background_labels in the dense heads (#3221) * Clean background_labels in the dense heads * clean format * Reformat * Fix CI bug * fix fsaf and the comments * fix newly added heads * fix yolact * complete comments * fix rpn transforming bug in two stage networks (#3754) * fix rpn transforming bug in two_stage * [Refactor] refactor get_subset_by_classes in dataloader for training with empty-GT images (#3695) * Add regression test and test data * Add fix for empty gt images * Trigger CI build * refactor get_subset_by_classes * reformat and fix docstring * [refactor]: move get_subset_by_classes to _filter_imgs * make img_ids consistent * resolve comments * simplify logics * add warning * add warning * add warning Co-authored-by: mmeendez8 <miguelmndez@gmail.com> * Fix nonzero in NMS for PyTorch 1.6.0 (#3867) * Support TTA of ATSS, FCOS, YOLOv3 (#3844) * Support TTA of ATSS, FCOS, YOLOv3 * Add comment * [Docs] Fix typo in docs/tutorials/new_dataset.md (#3876) * [Docs] Remove duplicate content in docs/config.md (#3875) * [Enhance]: Convert mask to bool before using it as img's index for robustness and speedup (#3870) * Convert mask to bool before using it as img's index * lint * Rewrite code in more concise way * Fix typo in bbox_flip (#3886) * fix the API change bug of PAA (#3883) * fix cv2 import error of ligGL.so.1 (#3891) * [enhance]: Improve documentation of modules and dataset customization (#3821) * update tutorials for changing training settings * update * Update details of hook * resolve comments * reformat * Update module documentations * add notes for classes specification * support to use pytorch 1.6 in docker (#3905) * support to use pytorch 1.6 in docker * update mmcv requirement * update docs * Add missing notes in data customization (#3906) * [Fix]: fix mask rcnn training stuck problem when there is no positive rois (#3713) * Fix mask rcnn stuck problem when there is no positive rois * support non pos inference in cascade methods, link CU-49tawu * print mmcv version in CI * use mmcv repo to check wrappers * change cpu build * upgrade mmcv requirements and change ci back * use pre-built whl in CI * Bump to v2.5.0 (#3879) * bump to v2.5.0 * update PR * resolve comments * clean unnecessary modification * update bc breaking * rephrase bc-breaking * rephrase * Added `generate_inputs_and_wrap_model` function for pytorch2onnx (#3857) * added `generate_inputs_and_wrap_model` function for pytorch2onnx * undo debugging changes * added model.cpu().eval() * refactored onnx * fix verify * move `generate_inputs_and_wrap_model` into mmdet.core.export * use `generate_inputs_and_wrap_model` in tools/pytorch2onnx.py * typo (#3917) * Edit mmdet.core.export docstring (#3912) * edit mmdet.core.export docstring * update doctest * fix small bugs * supports for HungarianMatchAssigner, add bbox_cxcywh_to_xyxy and bbox_xyxy_to_cxcywh for bbox transform * format box-wise related giou calculating as a function and implement it in iou2d_calculater.py * supports for BboxGIoU2D and re-implements giou_loss using bbox_gious * remove unnecessary * reformat * reformat docstring * reformat * rename * supports for giou calculating in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps * fix sabl validating bug in cascade_rcnn (#3913) * reformat * move giou related unit test from test_losses.py to test_iou2d_calculator.py * reformat * Avoid division by zero in PAA head when num_pos=0 * [Fix]: Avoid division by zero in PAA head when num_pos=0 (#3938) * explicitly add mode in giou_loss * Add supports for giou calculation in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps (#3936) * supports for BboxGIoU2D and re-implements giou_loss using bbox_gious * remove unnecessary * reformat * reformat docstring * reformat * rename * supports for giou calculating in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps * reformat * move giou related unit test from test_losses.py to test_iou2d_calculator.py * reformat * Avoid division by zero in PAA head when num_pos=0 * explicitly add mode in giou_loss * Add supports for giou calculation in BboxOverlaps2D, and add iou_calculator in hungarian_assigner * rename hungarian_match_assigner as hungarian_assigner * fix init * reformat docstring * Avoid division by zero in PAA head when num_pos=0 * fix cpu (#3948) * add mode for iou_calculator and make giou cost as a default case * make mode as a param in iou_calculator * reformat docsting * make iou_mode outside of iou_calculator Co-authored-by: Cao Yuhang <yhcao6@gmail.com> Co-authored-by: yuzhj <31239926+yuzhj@users.noreply.github.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> Co-authored-by: mmeendez8 <miguelmndez@gmail.com> Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com> Co-authored-by: LabMemNo003 <LabMemNo003@gmail.com> Co-authored-by: Ye Liu <yeliudev@outlook.com> Co-authored-by: aboettcher <aboettcher@users.noreply.github.com> Co-authored-by: Ryan Li <xlide@connect.ust.hk> Co-authored-by: Jiangmiao Pang <pangjiangmiao@gmail.com> * Re-implements RandomCrop to support different crop_type (#4093) * re-implements RandomCrop to support different crop_type * add assertion for relative related crop_type * reformat, and add image shape assertion for unit test * reformat * Supports for DETR training mode in process. (#3963) * add input_img_shape in img_meta, and supports training mode toy * toy commit * toy commit * re-organize and support for detr training mode * add unit test for detr training, testing, and transformer_head * add more unit test for transformer * reformat * remove unnecessary * use small transformer model for unit test * reformat docstring * reformat * reformat * Supports DETR e150 config (#4197) * rename e300 config as e150 config * reformat * add comments on override option in Resize * add comments on override option in Resize * position embeddingto positional encoding * fix unit tests * fix registry name bug * rename file Co-authored-by: v-qjqs <qjliqiaofei@gmail.com> Co-authored-by: Qiaofei Li <34116221+v-qjqs@users.noreply.github.com> Co-authored-by: Cao Yuhang <yhcao6@gmail.com> Co-authored-by: yuzhj <31239926+yuzhj@users.noreply.github.com> Co-authored-by: mmeendez8 <miguelmndez@gmail.com> Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com> Co-authored-by: LabMemNo003 <LabMemNo003@gmail.com> Co-authored-by: Ye Liu <yeliudev@outlook.com> Co-authored-by: aboettcher <aboettcher@users.noreply.github.com> Co-authored-by: Ryan Li <xlide@connect.ust.hk> Co-authored-by: Jiangmiao Pang <pangjiangmiao@gmail.com>
ZwwWayne
added a commit
that referenced
this pull request
Nov 29, 2020
* supports for DETR transformer * unit test with small data to avoid out of memory in CI * use batch size 1 for unit test to avoid out of memory * move transformer into utils folder and use more small data for unit test * reformat docstring * add more detailed docstring * reforamt * reformat and add build_transformer (#3866) * Supports for DETR position embedding (#3850) * supports for DETR position embedding * reformat docstring * reformat * move transformer_head folder into utils * remove unnecessary comments * reformat * reformat * reformat docstring * reformat * add optional for default parameters * add comments for unit test * avoid inplace in MultiheadAttention and add optional add_residual for FFN * Supports for DETR inference (#3941) * fix forward in FFN * supports detr inference mode with batchsize 1 * add docstring for DETR in detr.py * reformat and add doctest for TransformerHead * reformat, and move the mask construction from simple_test in detr.py to the forward process of transformer_head * rename detr_r50_1x_coco as detr_r50_8x4_1x_coco in config * reformat * mask input_img_shape in img_meta as a general modification of base_detector forward_test * reformat * Supports for DETR hungarian matcher. (#3929) * Add doc of modify loss (#3777) * add doc of modify loss * minor fix * change name * improve the function of simple_test_bboxes (#3853) * update * fix lint error * add bbox_pred_split in sabl_head * fix lint error * fix yapf * fix build error * fix yapf * Clean background_labels in the dense heads (#3221) * Clean background_labels in the dense heads * clean format * Reformat * Fix CI bug * fix fsaf and the comments * fix newly added heads * fix yolact * complete comments * fix rpn transforming bug in two stage networks (#3754) * fix rpn transforming bug in two_stage * [Refactor] refactor get_subset_by_classes in dataloader for training with empty-GT images (#3695) * Add regression test and test data * Add fix for empty gt images * Trigger CI build * refactor get_subset_by_classes * reformat and fix docstring * [refactor]: move get_subset_by_classes to _filter_imgs * make img_ids consistent * resolve comments * simplify logics * add warning * add warning * add warning Co-authored-by: mmeendez8 <miguelmndez@gmail.com> * Fix nonzero in NMS for PyTorch 1.6.0 (#3867) * Support TTA of ATSS, FCOS, YOLOv3 (#3844) * Support TTA of ATSS, FCOS, YOLOv3 * Add comment * [Docs] Fix typo in docs/tutorials/new_dataset.md (#3876) * [Docs] Remove duplicate content in docs/config.md (#3875) * [Enhance]: Convert mask to bool before using it as img's index for robustness and speedup (#3870) * Convert mask to bool before using it as img's index * lint * Rewrite code in more concise way * Fix typo in bbox_flip (#3886) * fix the API change bug of PAA (#3883) * fix cv2 import error of ligGL.so.1 (#3891) * [enhance]: Improve documentation of modules and dataset customization (#3821) * update tutorials for changing training settings * update * Update details of hook * resolve comments * reformat * Update module documentations * add notes for classes specification * support to use pytorch 1.6 in docker (#3905) * support to use pytorch 1.6 in docker * update mmcv requirement * update docs * Add missing notes in data customization (#3906) * [Fix]: fix mask rcnn training stuck problem when there is no positive rois (#3713) * Fix mask rcnn stuck problem when there is no positive rois * support non pos inference in cascade methods, link CU-49tawu * print mmcv version in CI * use mmcv repo to check wrappers * change cpu build * upgrade mmcv requirements and change ci back * use pre-built whl in CI * Bump to v2.5.0 (#3879) * bump to v2.5.0 * update PR * resolve comments * clean unnecessary modification * update bc breaking * rephrase bc-breaking * rephrase * Added `generate_inputs_and_wrap_model` function for pytorch2onnx (#3857) * added `generate_inputs_and_wrap_model` function for pytorch2onnx * undo debugging changes * added model.cpu().eval() * refactored onnx * fix verify * move `generate_inputs_and_wrap_model` into mmdet.core.export * use `generate_inputs_and_wrap_model` in tools/pytorch2onnx.py * typo (#3917) * Edit mmdet.core.export docstring (#3912) * edit mmdet.core.export docstring * update doctest * fix small bugs * supports for HungarianMatchAssigner, add bbox_cxcywh_to_xyxy and bbox_xyxy_to_cxcywh for bbox transform * format box-wise related giou calculating as a function and implement it in iou2d_calculater.py * supports for BboxGIoU2D and re-implements giou_loss using bbox_gious * remove unnecessary * reformat * reformat docstring * reformat * rename * supports for giou calculating in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps * fix sabl validating bug in cascade_rcnn (#3913) * reformat * move giou related unit test from test_losses.py to test_iou2d_calculator.py * reformat * Avoid division by zero in PAA head when num_pos=0 * [Fix]: Avoid division by zero in PAA head when num_pos=0 (#3938) * explicitly add mode in giou_loss * Add supports for giou calculation in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps (#3936) * supports for BboxGIoU2D and re-implements giou_loss using bbox_gious * remove unnecessary * reformat * reformat docstring * reformat * rename * supports for giou calculating in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps * reformat * move giou related unit test from test_losses.py to test_iou2d_calculator.py * reformat * Avoid division by zero in PAA head when num_pos=0 * explicitly add mode in giou_loss * Add supports for giou calculation in BboxOverlaps2D, and add iou_calculator in hungarian_assigner * rename hungarian_match_assigner as hungarian_assigner * fix init * reformat docstring * Avoid division by zero in PAA head when num_pos=0 * fix cpu (#3948) * add mode for iou_calculator and make giou cost as a default case * make mode as a param in iou_calculator * reformat docsting * make iou_mode outside of iou_calculator Co-authored-by: Cao Yuhang <yhcao6@gmail.com> Co-authored-by: yuzhj <31239926+yuzhj@users.noreply.github.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> Co-authored-by: mmeendez8 <miguelmndez@gmail.com> Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com> Co-authored-by: LabMemNo003 <LabMemNo003@gmail.com> Co-authored-by: Ye Liu <yeliudev@outlook.com> Co-authored-by: aboettcher <aboettcher@users.noreply.github.com> Co-authored-by: Ryan Li <xlide@connect.ust.hk> Co-authored-by: Jiangmiao Pang <pangjiangmiao@gmail.com> * Re-implements RandomCrop to support different crop_type (#4093) * re-implements RandomCrop to support different crop_type * add assertion for relative related crop_type * reformat, and add image shape assertion for unit test * reformat * Supports for DETR training mode in process. (#3963) * add input_img_shape in img_meta, and supports training mode toy * toy commit * toy commit * re-organize and support for detr training mode * add unit test for detr training, testing, and transformer_head * add more unit test for transformer * reformat * remove unnecessary * use small transformer model for unit test * reformat docstring * reformat * reformat * Supports DETR e150 config (#4197) * rename e300 config as e150 config * reformat * add comments on override option in Resize * add comments on override option in Resize * position embeddingto positional encoding * fix unit tests * fix registry name bug * rename file * remove unnecessary Co-authored-by: ZwwWayne <wayne.zw@outlook.com> Co-authored-by: Cao Yuhang <yhcao6@gmail.com> Co-authored-by: yuzhj <31239926+yuzhj@users.noreply.github.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> Co-authored-by: mmeendez8 <miguelmndez@gmail.com> Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com> Co-authored-by: LabMemNo003 <LabMemNo003@gmail.com> Co-authored-by: Ye Liu <yeliudev@outlook.com> Co-authored-by: aboettcher <aboettcher@users.noreply.github.com> Co-authored-by: Ryan Li <xlide@connect.ust.hk> Co-authored-by: Jiangmiao Pang <pangjiangmiao@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR reformat the merged version of DETR transformer.