replace RetinaNet with YOLOv5

[xjsxujingsong]

Hi, I tried to replace RetinaNet with YOLOv5 to get faster speed. The traning loss can decrease but mAP is always low. It cannot detect anything. Before replacement, I can sucessfully get 93% mAP using the default setting (RetinaNet). Could you please tell me if I miss anything here? I read the code about anchor part. It looks like only rectangle anchor is used, but i did not notice where is the way to resize the predicted box to original image corrdinate, and what is the purpose of weights in class BoxCoder
weights=(10., 10., 10., 5., 15.)

Here is the part I changed. I make anthor copy of yolo.py in YOLOv5/models. Only change the Detect class to replace cls head and reg head in RetinaNet. I also changed in anchor.py to reduce the pyramid level from 5 to 3 as in YOLOv5. self.pyramid_levels = [3, 4, 5]

class Detect(nn.Module):
def init(self, nc=80, anchors=(), ch=()): # detection layer
super(Detect, self).init()
self.nc = nc + 1 # number of classes
self.nr = 5 #xywha
self.na = 3 #len(anchors[0]) // 2 # number of anchors
self.cls_head = nn.ModuleList(nn.Conv2d(x, self.na * self.nc, 3, 1, 1) for x in ch) # output conv
self.reg_head = nn.ModuleList(nn.Conv2d(x, self.na * self.nr, 3, 1, 1) for x in ch) # output conv
self.init_weights()

def init_weights(self):
    prior = 0.01
    for m in self.cls_head.modules():
        if isinstance(m, nn.Conv2d):
            m.weight.data.fill_(0)
            m.bias.data.fill_(-math.log((1.0 - prior) / prior))
    for m in self.reg_head.modules():
        if isinstance(m, nn.Conv2d):
            m.weight.data.fill_(0)
            m.bias.data.fill_(0)

def get_cls_feature(self, index, x):
    x = torch.sigmoid(self.cls_head[index](x))
    x = x.permute(0, 2, 3, 1)
    n, w, h, c = x.shape
    x = x.reshape(n, w, h, self.na, self.nc)
    return x.reshape(x.shape[0], -1, self.nc)
def get_reg_feature(self, index, x):
    x = self.reg_head[index](x)
    x = x.permute(0, 2, 3, 1)
    return x.reshape(x.shape[0], -1, self.nr)
def forward(self, x):
    cls_score = torch.cat([self.get_cls_feature(idx, feature) for idx, feature in enumerate(x)], dim=1)
    bbox_pred = torch.cat([self.get_reg_feature(idx, feature) for idx, feature in enumerate(x)], dim=1)
    return [cls_score, bbox_pred]

I define another class to replace class RetinaNet, by only change a few lines.

class YOLOV5(nn.Module):
def init(self, cfg='yolov5l.yaml', ch=3, nc=None, export=False):
super(YOLOV5, self).init()
self.yolov5 = Model(cfg)
self.num_classes = 1 + 1 # class number + 1, I only have one class in the dataset
self.anchor_generator = Anchors(
ratios = np.array([0.5,1,2]),
)
self.num_anchors = self.anchor_generator.num_anchors
self.loss = IntegratedLoss(func='smooth')
self.box_coder = BoxCoder()

def forward(self, ims, gt_boxes=None, test_conf=None,process=None):
    anchors_list, offsets_list, cls_list, var_list = [], [], [], []
    original_anchors = self.anchor_generator(ims)   # (bs, num_all_achors, 5)
    anchors_list.append(original_anchors)
    # features = self.fpn(self.ims_2_features(ims))
    # cls_score = torch.cat([self.cls_head(feature) for feature in features], dim=1)
    # bbox_pred = torch.cat([self.reg_head(feature) for feature in features], dim=1)
    [cls_score, bbox_pred] = self.yolov5(ims)

..........the other parts are the same.

[ming71]

Hello, your effect on RetinaNet can reach the mAP of 93%, but not good for modified yolov5. It seems to be an incompatibility between the part of your code and the source code here.

1. The default setting only uses horizontal anchors, which is consistent with the results of the paper. Of course, rotated anchors are also optional, for example:Anchors(ratios = np.array([0.5,1,2]),rotations = np.array([0, 60, 120]) .
2. Code to resize the predicted box to original image coordinate can be found here.
3. Purpose of weights in class BoxCoder weights is used to balance the different items for box regression.
4. You can contact Shimadaaaaa, who is implementing DAL methods based on yolov5.
   Hope the above suggestions are helpful to you.

[xjsxujingsong]

Thanks for quick reply. I understand there must be something wrong with my code because yolov5 cannot detect anything. I will contact Shimadaaaaa for more details.
BTW, will rotated anchor improve the performance or not?

[xjsxujingsong]

Do you know the contact details of Shimadaaaaa? Could not find any way to contact

[ming71]

Preset more rotated anchors can achieve further improvement in my experiment (not shown in the paper).

[zhujingsong]

Thanks for quick reply. I understand there must be something wrong with my code because yolov5 cannot detect anything. I will contact Shimadaaaaa for more details.
BTW, will rotated anchor improve the performance or not?

Have you solved the problem? @xjsxujingsong