Supports for DETR in MMDetection. #4201
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…to transformer
Supports for DETR transformer
* supports for DETR position embedding * reformat docstring * reformat * move transformer_head folder into utils * remove unnecessary comments * reformat * reformat * reformat docstring * reformat * add optional for default parameters * add comments for unit test * avoid inplace in MultiheadAttention and add optional add_residual for FFN
* fix forward in FFN * supports detr inference mode with batchsize 1 * add docstring for DETR in detr.py * reformat and add doctest for TransformerHead * reformat, and move the mask construction from simple_test in detr.py to the forward process of transformer_head * rename detr_r50_1x_coco as detr_r50_8x4_1x_coco in config * reformat * mask input_img_shape in img_meta as a general modification of base_detector forward_test * reformat
* Add doc of modify loss (open-mmlab#3777) * add doc of modify loss * minor fix * change name * improve the function of simple_test_bboxes (open-mmlab#3853) * update * fix lint error * add bbox_pred_split in sabl_head * fix lint error * fix yapf * fix build error * fix yapf * Clean background_labels in the dense heads (open-mmlab#3221) * Clean background_labels in the dense heads * clean format * Reformat * Fix CI bug * fix fsaf and the comments * fix newly added heads * fix yolact * complete comments * fix rpn transforming bug in two stage networks (open-mmlab#3754) * fix rpn transforming bug in two_stage * [Refactor] refactor get_subset_by_classes in dataloader for training with empty-GT images (open-mmlab#3695) * Add regression test and test data * Add fix for empty gt images * Trigger CI build * refactor get_subset_by_classes * reformat and fix docstring * [refactor]: move get_subset_by_classes to _filter_imgs * make img_ids consistent * resolve comments * simplify logics * add warning * add warning * add warning Co-authored-by: mmeendez8 <miguelmndez@gmail.com> * Fix nonzero in NMS for PyTorch 1.6.0 (open-mmlab#3867) * Support TTA of ATSS, FCOS, YOLOv3 (open-mmlab#3844) * Support TTA of ATSS, FCOS, YOLOv3 * Add comment * [Docs] Fix typo in docs/tutorials/new_dataset.md (open-mmlab#3876) * [Docs] Remove duplicate content in docs/config.md (open-mmlab#3875) * [Enhance]: Convert mask to bool before using it as img's index for robustness and speedup (open-mmlab#3870) * Convert mask to bool before using it as img's index * lint * Rewrite code in more concise way * Fix typo in bbox_flip (open-mmlab#3886) * fix the API change bug of PAA (open-mmlab#3883) * fix cv2 import error of ligGL.so.1 (open-mmlab#3891) * [enhance]: Improve documentation of modules and dataset customization (open-mmlab#3821) * update tutorials for changing training settings * update * Update details of hook * resolve comments * reformat * Update module documentations * add notes for classes specification * support to use pytorch 1.6 in docker (open-mmlab#3905) * support to use pytorch 1.6 in docker * update mmcv requirement * update docs * Add missing notes in data customization (open-mmlab#3906) * [Fix]: fix mask rcnn training stuck problem when there is no positive rois (open-mmlab#3713) * Fix mask rcnn stuck problem when there is no positive rois * support non pos inference in cascade methods, link CU-49tawu * print mmcv version in CI * use mmcv repo to check wrappers * change cpu build * upgrade mmcv requirements and change ci back * use pre-built whl in CI * Bump to v2.5.0 (open-mmlab#3879) * bump to v2.5.0 * update PR * resolve comments * clean unnecessary modification * update bc breaking * rephrase bc-breaking * rephrase * Added `generate_inputs_and_wrap_model` function for pytorch2onnx (open-mmlab#3857) * added `generate_inputs_and_wrap_model` function for pytorch2onnx * undo debugging changes * added model.cpu().eval() * refactored onnx * fix verify * move `generate_inputs_and_wrap_model` into mmdet.core.export * use `generate_inputs_and_wrap_model` in tools/pytorch2onnx.py * typo (open-mmlab#3917) * Edit mmdet.core.export docstring (open-mmlab#3912) * edit mmdet.core.export docstring * update doctest * fix small bugs * supports for HungarianMatchAssigner, add bbox_cxcywh_to_xyxy and bbox_xyxy_to_cxcywh for bbox transform * format box-wise related giou calculating as a function and implement it in iou2d_calculater.py * supports for BboxGIoU2D and re-implements giou_loss using bbox_gious * remove unnecessary * reformat * reformat docstring * reformat * rename * supports for giou calculating in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps * fix sabl validating bug in cascade_rcnn (open-mmlab#3913) * reformat * move giou related unit test from test_losses.py to test_iou2d_calculator.py * reformat * Avoid division by zero in PAA head when num_pos=0 * [Fix]: Avoid division by zero in PAA head when num_pos=0 (open-mmlab#3938) * explicitly add mode in giou_loss * Add supports for giou calculation in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps (open-mmlab#3936) * supports for BboxGIoU2D and re-implements giou_loss using bbox_gious * remove unnecessary * reformat * reformat docstring * reformat * rename * supports for giou calculating in BboxOverlaps2D, and re-implements giou_loss using bbox_overlaps * reformat * move giou related unit test from test_losses.py to test_iou2d_calculator.py * reformat * Avoid division by zero in PAA head when num_pos=0 * explicitly add mode in giou_loss * Add supports for giou calculation in BboxOverlaps2D, and add iou_calculator in hungarian_assigner * rename hungarian_match_assigner as hungarian_assigner * fix init * reformat docstring * Avoid division by zero in PAA head when num_pos=0 * fix cpu (open-mmlab#3948) * add mode for iou_calculator and make giou cost as a default case * make mode as a param in iou_calculator * reformat docsting * make iou_mode outside of iou_calculator Co-authored-by: Cao Yuhang <yhcao6@gmail.com> Co-authored-by: yuzhj <31239926+yuzhj@users.noreply.github.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> Co-authored-by: mmeendez8 <miguelmndez@gmail.com> Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com> Co-authored-by: LabMemNo003 <LabMemNo003@gmail.com> Co-authored-by: Ye Liu <yeliudev@outlook.com> Co-authored-by: aboettcher <aboettcher@users.noreply.github.com> Co-authored-by: Ryan Li <xlide@connect.ust.hk> Co-authored-by: Jiangmiao Pang <pangjiangmiao@gmail.com>
* add input_img_shape in img_meta, and supports training mode toy * toy commit * toy commit * re-organize and support for detr training mode * add unit test for detr training, testing, and transformer_head * add more unit test for transformer * reformat * remove unnecessary * use small transformer model for unit test * reformat docstring * reformat * reformat
* rename e300 config as e150 config * reformat
ZwwWayne
approved these changes
Nov 29, 2020
ZwwWayne
reviewed
Nov 29, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements DETR in MMDetection. Duplicated of PR #3955.
The trained checkpoints for 150 epochs with batch size 16 (8x2) under one V100 mechine was released here, and the performance at epoch 150 is 40.1 mAP (vs 39.5 here in official DETR repo), which is 0.6 mAP higher in MMDetection.
The trained checkpoints for another training setting will be released soon.
Note currently only bounding box detection is implemented while the instance segmentation related task is not supported yet.