Question about post-processing of inference result

[kodaig]

Dear Mr. Ron

In test_emsemble.py at line 169 and 170, result image is resized to half and then doubled the size.
I think this operation should usually degrade the image quality.
Would you tell me why this operation is added to post-processing?

Best,
Kodai

[rlct1]

Dear Kodai,

Yes. The operation will probably degrade the image quality in terms of the visual quality (i.e., perhaps blurry results). The operation was added to post-processing as I found that it gives better results in terms of some objective metrics (i.e. PSNR and SSIM) for the AIM challenge.

I would say that the need for the operation depends on your applications. It would be better if you can try to add it and also eliminate it to see the performance on your own data.

Best,
Ron

[kodaig]

It is an interesting phenomenon. Do you have any insight on why those metrics have improved?

Thank you for your advice. I will try to figure out which is suitable for my purpose.

[rlct1]

Actually, during the training, the losses we used can be divided into two groups, one is per-pixel loss (e.g. L1, L2, MAE, etc.) and another is perceptual loss (e.g. style loss, perceptual loss, etc.). You may think that if your L1 is lower, you can get a higher PSNR. However, L1 tends to give blurry results as blurry results usually have a lower mean error as compared to those sharper results (if the sharp details are not correct, they tend to give higher per-pixel errors).

In my opinion, down-sample then up-sample is a way to make the results blurrier as the generated sharp details may not be very "correct". Hence, the operation can improve a bit in terms of PSNR and/or SSIM.

Yes. This is quite interesting. We are still working on finding a good balance between per-pixel accuracy and visual quality.

Thank you very much.

[kodaig]

Thank you very much for detailed explanation.
It really helped my understanding.