Good degradation configure for high resolution images

[ramdhan1989]

Hi,
my images are high resolution images with size around 5000 and 5000 (h and w). I run Degrade Generate but is very slows since the boxsize is relatively small. do you have suggestion about degradation configuration that fit better for my images? in term of speed and final performance.

please advice

thanks

[WindVChen]

@ramdhan1989 Hi, sorry for reply late. From your descriptions, the reason why degraded images generate slowly mainly comes from the input images' high resolution. The current code generates the degraded images in a per-pixel way, i.e., calculate the mean value of neighborhood for each pixel, thus with the size of input images increase, the time cost grows quadratically. Take the 5000 X 5000 image you use as an example, it is ~100 times bigger than 512 X 512 images and may take about ~100 times longer to generate a degraded image.

To speedup the generation, you can set the process number to more than 1 in the code to speedup the generation. However, I actually not so recommend you to directly use 5000 X 5000 as inputs. Even if you finish the degraded images, when you feed the images into the network, it may easily raise out of memory error, since the images are too big. Thus, if it is available, I recommend you to either cut the original images into small sub-patches (more recommended), or directly resize the original images (this way may cause the small objects loss in high-resolution images, thus not so recommended). Then, you can generate the degraded images of lower resolution inputs and the speed is of course faster.

As for the setting of degradation configuration, it differs according to different datasets. Take our LEVIR-Ship dataset as an example, most ship samples cover area under 20 X 20 pixels. We want to highlight the foregrounds/ships while blur the backgrounds, so the ships area should not degrade, i.e., the "boxSize" in DegradeGenerate.py should be 0 for ship area. As boxSize = KernelSize // 2, we should ensure the KernelSize lower than 2 in the ship area. Here, we consider the ship area as the area 20 pixels around the center of ship target, which is sufficient to cover the ship samples and ensure the ship not be blurred. Then as KernelSize = F(x) and F(x) = lambda^x, from the following figure (y=1.03^x), we can see when set lambda = 1.03, it can satisfy the need.

You can follow the above example to modify the lambda value according to your dataset. As for the value minDis = 130 * 130 in the code, it is to bound the degradation intensity, you should also modify it according to your dataset. Maybe just set it in the range (ship area < minDis < 1/4 image size).

[ramdhan1989]

Thank you so much for detail explanation. I will slice images into small patches 416416. after trained with small patches of image, how to do inference during implementation phase? do I need to resize 50005000 to 416*416? is there a mechanism to do something like SAHI (Slicing Aided Hyper Inference) ? Thanks

[WindVChen]

@ramdhan1989 Directly resize is not recommended, as it may loss some small targets and also not consistent with training phase (training with high spatial resolution, while testing with lower). A direct way is just to slice the 5000 X 5000 image into 416 X 416 patches, do inference per patch, and at last merge them, but in this way the output bbox may be not that accurate for the targets across the patches border. While, the vision example in SAHI displays that SAHI can also merge the bboxes across the patches border, thus it may be better to leverage SAHI than the former direct way (though I'm not so familiar with SAHI).

[ramdhan1989]

SAHI works with yolov5. DRENet seems can work with yolov5 as well according to .yaml inside the model folder. I would like to have your insight if yolov5 trained using the approach in this repo would work perfectly as well with SAHI? would you mind suggesting me which configuration/model (inside model folder) that works better?

[WindVChen]

It seems that SAHI is a inference strategy and have no much relation with the training phase, thus if it can work with YOLOv5, it should also work with DRENet, which is heavily based on YOLOv5 code structure. For the configuration, I may suggest to try DRENet.yaml, Ablation_Only_Enhancer.yaml, and Ablation_Only_Detector.yaml, and make comparisons to choose the best. As DRENet is originally designed for small targets in medium resolution image, the performance for large targets in high-resolution image is not verified, and you may refer to the Section V-C-5) in our paper for some information.

[ramdhan1989]

Thanks for the suggestion. I sliced the images into medium size 416 X 416. Hopefully, by combining with SAHI, they can give good prediction on high resolution image. I will inform about the result.

[ramdhan1989]

hi, thanks for your help. I was successful to combine the trained model using this repo with SAHI. thanks again for your guidance.