Detector - Mosaic data augmentation

[AlexeyAB]

Mosaic-data augmentation (like CutMix augmentation): #3114 (comment)

• Mosaic-data mAP increasing: Implement Yolo-LSTM (~+4-9 AP) for detection on Video with high mAP and without blinking issues #3114 (comment)

• comparison MixUp and CutMix: BoF (Bag of Freebies) - Visually Coherent Image Mixup ~+4 AP@[.5, .95] #3272 (comment)

Use:

[net]
mosaic=1

Currently works only if Darknet is compiled with OpenCV.

---

New mosaic:

---

Old mosaic=1 - CLICK ME to view

---

[AlexeyAB]

Implemented.

[Kyuuki93]

Something goes wrong with

[net]
mosaic=1

chart:

cfg file:
yolov3-spp-giou-gs-mos.cfg.txt

without mosaic chart:

[LukeAI]

For me, mosaic gave me -0.4% mAP - don't really know if that's just random fluctuation or it did actually hurt performance a little.

[AlexeyAB]

@LukeAI @Kyuuki93

I improved mosaic=1 for Detector. Try new mosaic.
Two commits: 87f36b7 and 13f064f

[Kyuuki93]

@LukeAI @Kyuuki93

I improved mosaic=1 for Detector. Try new mosaic.
Two commits: 87f36b7 and 13f064f

results were updated here #3874 (comment)

[AlexeyAB]

@Kyuuki93 It seems that new mosaic=1 significantly improves AP@75

• spp,giou,gs(iou_n=0.50,uc_n=0.50) | - |   | 91.39% AP@50 | 58.01% AP@75
• spp,giou,gs,mosaic(n=0.50,uc_n=0.50) | - |   | 90.48% AP@50 | 60.02% AP@75

Also it seems that iou_n=0.07 is too low value for GIoU, and iou_n=0.5 is much better.

[Kyuuki93]

@Kyuuki93 It seems that new mosaic=1 significantly improves AP@75

• spp,giou,gs(iou_n=0.50,uc_n=0.50) | - |   | 91.39% AP@50 | 58.01% AP@75
• spp,giou,gs,mosaic(n=0.50,uc_n=0.50) | - |   | 90.48% AP@50 | 60.02% AP@75

Also it seems that iou_n=0.07 is too low value for GIoU, and iou_n=0.5 is much better.

And I updated mosaic results iou_thresh = 0.213, the improvement at AP@75 still exists but not that significant

• spp,giou,gs(in=0.5,un=0.5,it=0.213) | 91.89% AP@50 | 63.53% AP@75|
• spp,giou,gs,mosaic(in=0.5,un=0.5,it=0.213) | 91.82% AP@50 | 63.89% AP@75|

[AlexeyAB]

@Kyuuki93 So both iou_thresh=0.213 and mosaic=1 increase accuracy.
Maybe mosaics will give a greater increase in accuracy on large datasets

[laclouis5]

For me, mosaic gave me -0.4% mAP - don't really know if that's just random fluctuation or it did actually hurt performance a little.

Speaking of training accuracy fluctuation I did an experiment on a small private dataset (see #3874 (comment) for dataset details).

I trained 15 instances of "Yolo v3 Tiny 3L" for 10_000 steps on the same dataset with same config file and parameters.

The randomness of training brings some fluctuation in accuracy (mAP@.5):

• extend (max - min mAP value) is 2.18%
• standard deviation is 0.62% around a mean of 82.18%

Std Dev and extend are higher for classes harder to detect.

[AlexeyAB]

@laclouis5

What pre-trained weights-file did you use for training yolo_v3_csresnext50-pan-spp.cfg there? #3874 (comment)

[laclouis5]

@AlexeyAB

I don't use any pre-trained weights in all my experiments, every network is trained from scratch.

[AlexeyAB]

@laclouis5 (I don't use any pre-trained weights) - This can reduce accuracy, the larger the model, the more pre-trained weights are needed.

[laclouis5]

@AlexeyAB

I try to train my networks in a very similar context to obtain a fair comparison. I don't fully control and understand how pre-trained weights affect initialisation and training as pre-trained weights could be different between networks (Tiny and plain Yolo for instance), so I prefer to train from scratch to compare apples to apples.

I also observe that on small database and small network (Yolo v3 Tiny) training curve is steeper but the final accuracy (mAP@.5) is roughly the same as it plateaus rapidly near 3_000 iterations in my case.

In the following image are overlapped training curves with and without pre-trained weights (from a network I previously trained).

Maybe pre-trained weights / transfert learning is more relevant and can increase accuracy for deeper networks such as yolo_v3_csresnext50-pan-spp.cfg and/or with bigger databases with more classes.

That being said, when deploying a network I would certainly train with Coco weights to get the best result.

[laclouis5]

@AlexeyAB I'll train yolo_v3_csresnext50-pan-spp.cfg with pre-trained weights to quantify the difference on this one.

[AlexeyAB]

@laclouis5

Maybe pre-trained weights / transfert learning is more relevant and can increase accuracy for deeper networks such as yolo_v3_csresnext50-pan-spp.cfg and/or with bigger databases with more classes.

I think so.

I'll train yolo_v3_csresnext50-pan-spp.cfg with pre-trained weights to quantify the difference on this one.

Yes, try and compare accuracy.

[tianfengyijiu]

Is ok to use mosaic=1 and mixup=1 at the same time? @AlexeyAB

[AlexeyAB]

@tianfengyijiu Yes, but will be used only Mosaic.

[laclouis5]

@AlexeyAB Here is the result:

Model	Training Chart	mAP@.5	mAP@[0.5...0.95]
Yolo v3 Spp Panet		83.38%	42.21%
Yolo v3 Spp Panet with pre-trained weights		88.34%	45.88%

The pre-trained version is much more stable, +0.30% mAP@0.5 and +3% mAP@[0.5...0.95] higher than my best non pre-trained network.

If I have time l'll train other networks with pre-trained weights to be able to compare this result against.

[BernoGreyling]

@AlexeyAB ,

Continuing issue with mosaic in Detector experienced here : #4432 (comment)

I'm indeed using the newest repo and the issue seems to only occur when mosaic is used with letter_box and images are positioned in such a way that they overlap awkwardly. All other images come out correctly.

For example :

This issue goes away when turning off letter_box as this prevents images running over one another, it seems. For example :

[AlexeyAB]

@BernoGreyling Thanks! Yes, it seems there is an issue for combination letter_box=1 mosaic=1

[brian208579]

hi @AlexeyAB
Is this problem solved ? ( combination letter_box=1 mosaic=1 )

[AlexeyAB]

Not yet

[EtheneXiang]

can you share the paper for mosaic, i want cite, but not find in Google Scholar, THX

[ald2004]

can you share the paper for mosaic, i want cite, but not find in Google Scholar, THX

mosaic cames from yolov3 or yolov4

[AlexeyAB]

https://arxiv.org/pdf/2004.10934.pdf

[calves91]

@BernoGreyling Thanks! Yes, it seems there is an issue for combination letter_box=1 mosaic=1

Hi @AlexeyAB
Has this bug been fixed yet?

[AlexeyAB]

@calves91

Yes,

Just use

[net]
mosaic=1
mosaic_bound=1
letter_box=1

---

In my experiments, usually this "bug" improves accuracy, so usually it is better to train without mosaic_bound=1, because in the most cases we need to detect the whole object:

• mosaic_bound=1 - increases accuracy for occluded objects (it detects only visible part of object)
• mosaic_bound=0 - increases accuracy for not-occluded objects (it detects visible+occluded parts of object) - usually it is more important, because we have much more not-occluded objects than occluded

---

https://github.com/AlexeyAB/darknet/wiki/CFG-Parameters-in-the-%5Bnet%5D-section

mosaic_bound=1 - limits the size of objects when mosaic=1 is used (does not allow bounding boxes to leave the borders of their images when Mosaic-data-augmentation is used)