data augmentation zoom

[LukeAI]

Is anybody aware of a script to augment my data by randomly cropping (zooming in on smaller objects) and adjusting the corresponding bounding boxes annotations accordingly?

[LukeAI]

I was wondering why this isn't part of the darknet augmentation? at least as a cfg option? Might it not be an effective way to help Yolo move towards scale invariance if it saw zoomed in versions of the smaller objects during training?

[AlexeyAB]

jitter= and random=1 in [yolo] layer do these things https://github.com/AlexeyAB/darknet/wiki/CFG-Parameters-in-the-different-layers

[LukeAI]

Ah, so I did read those:

jitter=.3 - randomly crops and resizes images with changing aspect ratio from x(1 - 2*jitter) to x(1 + 2*jitter) (data augmentation parameter is used only from the last layer)`

random=1 - randomly resizes network for each 10 iterations from 1/1.4 to 1.4(data augmentation parameter is used only from the last layer)

So random=1 will essentially blur the whole image - and jitter will randomly crop and also warp the aspect ratio. What are the constraints on the random cropping - how small might the crops go?

[AlexeyAB]

random=1 - will not blur image, it will resize the network (and then resize image to this network size)

yes, jitter= will randomly crop, resize and change aspect ratio.
For example, it can crop part of image (1 - 2*jitter) x (1 - 2*jitter), so if jitter=0.3 then
the smallest part of image which can be cropped 0.4 x 0.4, i.e. image size will be increased 2.5x times (width and height will be increased 2.5x times)

[LukeAI]

Is there any way that I could get greater cropping without drastically warping the image?

[AlexeyAB]

Do you want just change the size without changing aspect ratio?

[LukeAI]

Well, my dataset is fairly high resolution images - almost 4 times the width and height of the network and it's biased towards objects in the distance, but I want the model to be better at detecting objects closer up. Maybe jitter=0.4 would be ok for the purposes of cropping - I'm a little concerned that it may be too much distortion?

[Hwijune]

Rotation augmentation is also required.

[LukeAI]

My thinking is - for my dataset, the shapes of the objects never change, they are all rigid squares and rectangles, so jitter augmentation may be counterproductive if it stops the detector from being able to learn that. the zoom/crop effect does sound very useful though, I want the detector to be as scale-invariant as possible.

[AlexeyAB]

Now you can use resize=2.0 it will randomly resize and move image.

---

will resize image [1/2; 2]x

[yolo]
jitter=0.0
resize=2.0

---

will resize image [1/2; 1]x

[yolo]
jitter=0.0
resize=0.5

[LukeAI]

so to clarify: resize=0.5 will randomly crop away 50% of the area of each image?

[AlexeyAB]

resize=0.5 - image width and height will be reduced by 2 times (image area will be reduced by 4 times)

---

You can see how this works by usig -show_imgs flag at the end of training command.

[AlexeyAB]

Because you get Nan loss.
You are doing something wrong.

[AlexeyAB]

• What dataset did you use?

• Try to use -show_imgs flag, do you see correct bboxes?

• What training command did you use?

• Show such screenshot:
  

• Attach you cfg-file.

• Try to train with jitter=0.0 resize=1.5 for each of 3 [yolo] layers.

[LukeAI]

I was doing something wrong. I accidentally had burn in set too low. I set it back to default and got this: (baseline is around 76% mAP)

I'll try again with your suggested jitter and resize.

[AlexeyAB]

• If your training/validation/test images has the same size - use jitter=0.0 resize=1.5 and preferably letter_box=0

• If your training/validation/test images has different size - use letter_box=1 jitter=0.0 resize=1.5 or letter_box=0 jitter=0.3 resize=0.0

[LukeAI]

training/validation/test are the same size. Is letter_box=0 not the default behaviour?

[AlexeyAB]

letter_box=0 is the default behaviour

[LukeAI]

training with resize=1.5, mosaic=0, jitter=0, letter_box=0, random=0
AP is down from the ~73% AP that I got with default yolov4 at same size.

resize=1.5

baseline:

[AlexeyAB]

baseline:

What params resize=1.5, mosaic=0, jitter=0, letter_box=0, random=0 did you use?

It seems that your images size are not the same, or aspect ratio of objects are not the same.