Add GIoU loss into this repo? ~+3 AP@[.5, .95]

[TaihuLight]

YoloV3 with GIoU loss implemented in original Darknet as https://github.com/generalized-iou/g-darknet
Will GIoU loss be supported in this version of Darknet?

@article{Rezatofighi_2018_CVPR,
  author    = {Rezatofighi, Hamid and Tsoi, Nathan and Gwak, JunYoung and Sadeghian, Amir and Reid, Ian and Savarese, Silvio},
  title     = {Generalized Intersection over Union},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2019},
}

[AlexeyAB]

I will think about it:
https://arxiv.org/abs/1902.09630v2

https://giou.stanford.edu/

[AlexeyAB]

@TaihuLight Hi,

I added GIoU: 6e13527

from: https://github.com/generalized-iou/g-darknet

Cfg-files:

• https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov3-voc.yolov3-giou-40.cfg
• https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov3.coco-giou-12.cfg

[tdurand]

@AlexeyAB this is used for training or inference ?

[AlexeyAB]

@tdurand
It should be used for training.

[tdurand]

@tdurand thanks and sorry for this other question, I went through the paper and the website but failed to understand if those trained weight published here: https://github.com/generalized-iou/g-darknet#pre-trained-models improves inference speed / accuracy ? Or is it just a method to train faster ?

[AlexeyAB]

@tdurand As they said, the MS COCO AP@[.5, .95] was increased: https://arxiv.org/pdf/1902.09630v2.pdf

It means that by training with GIoU:

• AP@[.5, .95] - was increased
• mAP@0.75 - was increased
• mAP@0.5 - was decreased

---

Checking mAP on MS COCO 2014 validation dataset 5k.txt - you can get this dataset by using: https://github.com/AlexeyAB/darknet/blob/master/scripts/get_coco_dataset.sh

Yolo v3 (not spp) width=608 height=608: https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov3.coco-giou-12.cfg

Using these weights-files: https://github.com/generalized-iou/g-darknet#pre-trained-models

• MSE: https://stanford.io/2Hb6hpz
• GIoU: https://stanford.io/2PWP8Cz

---

mAP@0.75 - (IoU_threshold = 75%)

• GIoU - 35.05% mAP@0.75 - 0.351
  darknet.exe detector map cfg/coco.data cfg/yolov3.coco-giou-12.cfg yolov3_final_giou.weights -points 101 -iou_thresh 0.75

• MSE - 31.39% mAP@0.75 - 0.314
  darknet.exe detector map cfg/coco.data cfg/yolov3.coco-giou-12.cfg yolov3_492000_mse.weights -points 101 -iou_thresh 0.75

• MSE (default weights https://pjreddie.com/media/files/yolov3.weights ) - 31.63% mAP@0.75 - 0.316
  darknet.exe detector map cfg/coco.data cfg/yolov3.coco-giou-12.cfg yolov3.weights -points 101 -iou_thresh 0.75

Result is slightly different than in Pycoco-tool, since Pycoco-tool takes into account parameters crowd in MS COCO labels.

---

mAP@0.50 (IoU_threshold = 50%)

• GIoU - 52.17% mAP@0.50 - 0.522
  darknet.exe detector map cfg/coco.data cfg/yolov3.coco-giou-12.cfg yolov3_final_giou.weights -points 101 -iou_thresh 0.50

• MSE - 52.17% mAP@0.50 - 0.522
  darknet.exe detector map cfg/coco.data cfg/yolov3.coco-giou-12.cfg yolov3_492000_mse.weights -points 101 -iou_thresh 0.50

• MSE (default weights https://pjreddie.com/media/files/yolov3.weights ) - 55.19% mAP@0.50 - 0.552
  darknet.exe detector map cfg/coco.data cfg/yolov3.coco-giou-12.cfg yolov3.weights -points 101 -iou_thresh 0.50

Result is slightly different than in Pycoco-tool, since Pycoco-tool takes into account parameters crowd in MS COCO labels.

[tdurand]

many thanks !

[AlexeyAB]

Explanation of AP(GIoU)-metric: generalized-iou/g-darknet#12 (comment)

[LukeAI]

Is it possible to evaluation the GIoU metric with this repo?
Why is it that there is an increase in mAP and AP75 but a decrease in AP50 with GIoU loss?

[AlexeyAB]

@LukeAI

Is it possible to evaluation the GIoU metric with this repo?

Do you mean mAP@GIoU_treshold instead of mAP@IoU_treshold ?
No.

Why is it that there is an increase in mAP and AP75 but a decrease in AP50 with GIoU loss?

correct classifications are slightly less likely but the bounding boxes are tighter

[yrc08]

@AlexeyAB
I want to train https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov3.coco-giou-12.cfg, I can use https://pjreddie.com/media/files/darknet53.conv. 74 weight documents?

[AlexeyAB]

@yrc08 Yes.

[yrc08]

@AlexeyAB Thank you very much for your prompt reply.

…

------------------ 原始邮件 ------------------ 发件人: "Alexey"<notifications@github.com>; 发送时间: 2019年10月23日(星期三) 晚上6:47 收件人: "AlexeyAB/darknet"<darknet@noreply.github.com>; 抄送: "洋葱"<2990064358@qq.com>;"Mention"<mention@noreply.github.com>; 主题: Re: [AlexeyAB/darknet] Add GIoU loss into this repo? ~+3 AP@[.5,.95] (#3249) @yrc08 Yes. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.