Implement FCOS training tricks #2935

Chrisfsj2051 · 2020-06-08T06:31:56Z

Implement another three training tricks (except for center_sampling) used in the official repo (https://github.com/tianzhi0549/FCOS). By jointly using them, mAP gains of up to 4.9 (with DCN) and 1.3 (without DCN) will be brought.

In this implementation, the following results are achieved:
without DCN: mAP=38.6 (0.1 lower than the official repo)
with DCN: mAP=42.5 (0.2 higher than the official repo)

Train FCOS with four tricks (including center_sampling) arrording to official repo (https://github.com/tianzhi0549/FCOS). without DCN: mAP=38.6 (0.1 lower than official repo) with DCN : mAP=42.5 (0.2 higher than official repo)

configs/fcos/fcos_center-normbbox-centeronreg-giou_r50_caffe_fpn_gn-head_4x4_1x_coco.py

mmdet/models/dense_heads/fcos_head.py

yhcao6 · 2020-06-08T08:21:34Z

mmdet/models/dense_heads/fcos_head.py

                    norm_cfg=self.norm_cfg,
-                    bias=self.norm_cfg is None))
+                    bias=True))


Originally most of the time bias will be False since FCOS uses GN in the head. Could I ask if bias has a great effect on the performance?

yhcao6 · 2020-06-08T08:31:27Z

Thanks for your contribution, please check the comments!

Johnson-Wang

Please also load the original checkpoint to see if it is compatible.

configs/fcos/fcos_center-normbbox-centeronreg-giou_r50_caffe_fpn_gn-head_4x4_1x_coco.py

configs/fcos/fcos_center-normbbox-centeronreg-giou_r50_caffe_fpn_gn-head_dcn_4x4_1x_coco.py

mmdet/models/dense_heads/fcos_head.py

Johnson-Wang · 2020-06-08T07:21:54Z

mmdet/models/dense_heads/fcos_head.py

                    norm_cfg=self.norm_cfg,
-                    bias=self.norm_cfg is None))
+                    bias=True))


Is bias=True used in the original implementation? Actually, using bias before a normalization layer is not so standard, it's better to enable users to specify it.

Chrisfsj2051 · 2020-06-08T16:22:29Z

Hi @yhcao6 and @Johnson-Wang , thanks for your comments!

The origin setting and implementation of FCOS in mmdetection are slightly different from official repo. For example, the author set nms_th as 0.6 and doesn't use grad clip, while in mmdetection the nms_th is set as 0.5 and grad clip is used. Also, the official repo use bias=True in conv layer while the previous implementation in mmdetection doesn't. Here is the link to the official implementation of FCOSHead.

I implement and train this model by following the author, which may lead to differences from the previous implementation in mmdetection. I suggest to keep the bias setting for now. Also, I will conduct some experiments to see if it will raise significant difference, which will take about two days.

…n_4x4_1x_coco.py

Johnson-Wang · 2020-06-09T00:56:43Z

Please check your ci test.

Johnson-Wang · 2020-06-09T03:27:19Z

mmdet/models/dense_heads/fcos_head.py

+        cls_scores = []
+        bbox_preds = []
+        centernesses = []
+        for i, x in enumerate(feats):


Since there is no special operation on a specific fpn layer, is it possible to revert to forward_single implementation?

Except that there would be another argument (stride) to be passed to forward_single.

Great idea! Already updated in the latest commit.

mmdet/models/dense_heads/fcos_head.py

Johnson-Wang

The main implementation seems fine to me now. There is another work to do here. The unittest of fcos head was missing. It would be better to add unittest gradually in the future. Could you please add the unittest of fcos? (Special attention should be paid to corner cases such as bbox_ignore, empty gt_bbox et al. )

mmdet/models/dense_heads/fcos_head.py

Chrisfsj2051 · 2020-06-12T07:13:56Z

Hi @Chrisfsj2051, it's great to see the implementation of improved FCOS. I was interested the performance of FCOS if you just add the dconv2 layer to the last conv layers without using the dconv backbone. Thanks.

Sure, I will try this later.

Cool! Thank you.

Get mAP=39.9 without using DCN in backbone.

hyz-xmaster · 2020-06-12T07:18:14Z

Hi @Chrisfsj2051, it's great to see the implementation of improved FCOS. I was interested the performance of FCOS if you just add the dconv2 layer to the last conv layers without using the dconv backbone. Thanks.

Sure, I will try this later.

Cool! Thank you.

Get mAP=39.9 without using DCN in backbone.

Amazing. Thanks a lot.

Johnson-Wang · 2020-06-12T09:12:04Z

tests/test_heads.py

+    # random inputs
+    gt_bboxes = [
+        torch.Tensor([[23.6667, 23.8757, 238.6326, 151.8874]]),
+    ]


Does it support gt_bboxes_ignore?

Unit test doesn't include gt_bboxes_ignore test, since the implemenration of FCOS doesn't support gt_bboxes_ignore (passed as argument but not actually used).

mmdet/models/dense_heads/fcos_head.py

Johnson-Wang · 2020-06-12T15:36:40Z

Still, please have a check on the ci.

mmdet/models/dense_heads/fcos_head.py

hellock · 2020-06-16T15:29:27Z

mmdet/models/dense_heads/fcos_head.py

@@ -80,26 +126,31 @@ def _init_layers(self):
        self.reg_convs = nn.ModuleList()
        for i in range(self.stacked_convs):
            chn = self.in_channels if i == 0 else self.feat_channels
+            if self.dcn_on_last_conv and \
+                    i == self.stacked_convs - 1:


Use parentheses instead of backslashes.
if (self.dcn_on_last_conv and i == self.stacked_convs - 1) and let yapf to format it.

hellock · 2020-06-16T15:30:14Z

mmdet/models/dense_heads/fcos_head.py

+        centerness_on_reg (bool): If true, position centerness on the
+            regress branch. Please refer to https://github.com/tianzhi0549/FCOS/issues/89#issuecomment-516877042.
+        dcn_on_last_conv (bool): If true, use dcn in the last layer of
+            towers.


Default values of several arguments are not described.

Chrisfsj2051 · 2020-06-16T16:05:34Z

Hi @hellock , thanks for the comments!
Default values for some arguments have been specified currently. However, I didn't describe the other default values (regress_ranges, loss_cls, etc.) since they might be too long to be properly shown in docstring.

* 123 * implement FCOS with tricks (FCOS_plus) Train FCOS with four tricks (including center_sampling) arrording to official repo (https://github.com/tianzhi0549/FCOS). without DCN: mAP=38.6 (0.1 lower than official repo) with DCN : mAP=42.5 (0.2 higher than official repo) * fix * fix * reformatted * fix configs & reformat * add docstring * Update fcos_center-normbbox-centeronreg-giou_r50_caffe_fpn_gn-head_dcn_4x4_1x_coco.py * add noqa: E501 since the url is too long * fix flake * fix flake * fix yapf * fix * rewrite forward & reformat * reformat & add docstring * reformat * reformat * update docstring * TODO: add unit test Should be finished tomorrow * add unit test * fix * fix docstring * reformat * add docstring * reformat * Update fcos_head.py * Update fcos_head.py * Update fcos_head.py * Update fcos_head.py * Update fcos_head.py * Update test_heads.py * Update fcos_head.py * Update README.md

hedes1992 · 2020-09-25T07:55:54Z

Hello,
Thank you very much for your great work.
I just find some bugs(maybe) in the implementation which will not disturb the training process, but just the in-consistency...
In detail, in fcos_head.py, the distance2bbox function just calculate the predicting bbox and gt bbox in the original image. But if self.norm_on_bbox is True, like here,
the "pos_bbox_targets" is normed by corresponding stride, but the "pos_points" is in the original image.
In this way, the "pos_decoded_target_preds" is not the true gt bboxes... But this will not cause error, because the loss_bbox here is IoULoss(which just use the relative offset between pred and gt to calculate loss).
Is my worry is correct?

Co-authored-by: Tony Hu <tonyhu@amazon.com>

Chrisfsj2051 added 5 commits May 26, 2020 16:18

123

74919fb

Merge branch 'ups_master' into fcos_plus

032e944

implement FCOS with tricks (FCOS_plus)

7a5ba13

Train FCOS with four tricks (including center_sampling) arrording to official repo (https://github.com/tianzhi0549/FCOS). without DCN: mAP=38.6 (0.1 lower than official repo) with DCN : mAP=42.5 (0.2 higher than official repo)

fix

8b24c63

fix

c16c74e

hellock requested review from yhcao6 and Johnson-Wang June 8, 2020 06:48

reformatted

ce61e82

yhcao6 reviewed Jun 8, 2020

View reviewed changes

Johnson-Wang reviewed Jun 8, 2020

View reviewed changes

Chrisfsj2051 added 2 commits June 8, 2020 23:55

fix configs & reformat

ac71f85

add docstring

db9ef66

Update fcos_center-normbbox-centeronreg-giou_r50_caffe_fpn_gn-head_dc…

a72b963

…n_4x4_1x_coco.py

Chrisfsj2051 added 6 commits June 9, 2020 09:08

add noqa: E501 since the url is too long

d867d66

fix flake

88b689b

fix flake

8d1f07a

fix yapf

3b182d9

Merge branch 'ups_master' into fcos_plus

acb24db

fix

42f5ea9

Johnson-Wang reviewed Jun 9, 2020

View reviewed changes

rewrite forward & reformat

38548f9

Johnson-Wang reviewed Jun 9, 2020

View reviewed changes

mmdet/models/dense_heads/fcos_head.py Show resolved Hide resolved

mmdet/models/dense_heads/fcos_head.py Show resolved Hide resolved

Chrisfsj2051 added 3 commits June 9, 2020 14:15

reformat & add docstring

b0b8843

reformat

041d674

reformat

4a06292

Johnson-Wang reviewed Jun 9, 2020

View reviewed changes

update docstring

1a42401

fix

ce00bab

Johnson-Wang reviewed Jun 12, 2020

View reviewed changes

Chrisfsj2051 added 6 commits June 14, 2020 01:29

fix docstring

ab9c83e

reformat

2923ac5

add docstring

00d13ba

reformat

e4ca653

Update fcos_head.py

0cd6ddc

Update fcos_head.py

1bb790c

Johnson-Wang reviewed Jun 14, 2020

View reviewed changes

mmdet/models/dense_heads/fcos_head.py Outdated Show resolved Hide resolved

Chrisfsj2051 added 4 commits June 14, 2020 10:52

Update fcos_head.py

e970118

Update fcos_head.py

1ff5372

Update fcos_head.py

36f4e25

Update test_heads.py

080cf2a

Johnson-Wang approved these changes Jun 15, 2020

View reviewed changes

hellock reviewed Jun 16, 2020

View reviewed changes

Chrisfsj2051 added 2 commits June 16, 2020 23:35

Merge branch 'ups_master' into fcos_plus

202927d

Update fcos_head.py

40c9c42

Update README.md

d53edd0

hellock approved these changes Jun 16, 2020

View reviewed changes

hellock merged commit 565e070 into open-mmlab:master Jun 16, 2020

chunchet-ng mentioned this pull request Jul 22, 2021

Confusion about gt_bboxes_ignore #5668

Closed

zhangh-dev mentioned this pull request Jan 12, 2022

GN to BN in auto_assign head, mAP for coco drops 3%． #6991

Open

FANGAreNotGnu pushed a commit to FANGAreNotGnu/mmdetection that referenced this pull request Oct 23, 2023

Update to v0.7 (open-mmlab#2935)

2db0be3

Co-authored-by: Tony Hu <tonyhu@amazon.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement FCOS training tricks #2935

Implement FCOS training tricks #2935

Chrisfsj2051 commented Jun 8, 2020

yhcao6 Jun 8, 2020

yhcao6 commented Jun 8, 2020

Johnson-Wang left a comment

Johnson-Wang Jun 8, 2020

Chrisfsj2051 commented Jun 8, 2020 •

edited

Johnson-Wang commented Jun 9, 2020

Johnson-Wang Jun 9, 2020

Johnson-Wang Jun 9, 2020

Chrisfsj2051 Jun 9, 2020

Johnson-Wang left a comment

Chrisfsj2051 commented Jun 12, 2020

hyz-xmaster commented Jun 12, 2020

Johnson-Wang Jun 12, 2020

Chrisfsj2051 Jun 13, 2020

Johnson-Wang commented Jun 12, 2020

hellock Jun 16, 2020 •

edited

hellock Jun 16, 2020

Chrisfsj2051 commented Jun 16, 2020

hedes1992 commented Sep 25, 2020 •

edited

Implement FCOS training tricks #2935

Implement FCOS training tricks #2935

Conversation

Chrisfsj2051 commented Jun 8, 2020

Choose a reason for hiding this comment

yhcao6 commented Jun 8, 2020

Johnson-Wang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chrisfsj2051 commented Jun 8, 2020 • edited

Johnson-Wang commented Jun 9, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Johnson-Wang left a comment

Choose a reason for hiding this comment

Chrisfsj2051 commented Jun 12, 2020

hyz-xmaster commented Jun 12, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Johnson-Wang commented Jun 12, 2020

hellock Jun 16, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chrisfsj2051 commented Jun 16, 2020

hedes1992 commented Sep 25, 2020 • edited

Chrisfsj2051 commented Jun 8, 2020 •

edited

hellock Jun 16, 2020 •

edited

hedes1992 commented Sep 25, 2020 •

edited