How to reproduce the data of LPIPS distance?

[WorkingCC]

Your work gets surprising results and I expect to reproduce the data of LPIPS distance that you list in Figure6. Given one input image, you sample 19 outputs. For every input(maps), do you calculate the LPIPS distance between the given image(maps) and corresponding 19 samples(satellite) ? After that, you sum those 19 groups of data and have a average? Is it the same to other 99 input images in your experiment ? I'm confused about this and looking forward to your reply, thank you!

[richzhang]

For each input image, we produce 20 outputs and compute the distance between each pair of outputs, and take the average. We do this across input images, and take the average.

[WorkingCC]

@richzhang ,thank you, I get the point but still have some questions. Do you compute LPIPS distance between the input image(maps) and the sampled output, or between the corresponding real satellite image and the sampled output?

[richzhang]

Neither - between two random outputs

…

On Mon, Jun 25, 2018, 4:11 AM Congcong ***@***.***> wrote: @richzhang <https://github.com/richzhang> ,thank you, I get the point but still have some questions. Do you compute LPIPS distance between the input image(maps) and the sampled output, or between the corresponding real satellite image and the sampled output? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ARPfaqMGWIU5YB6725ilP33ELcuOjssBks5uAMVlgaJpZM4U1JpK> .

[WorkingCC]

For each input image, you sample two random codes Z and generated two random outputs at one time. And you do it up to 19 times, i.e., for each input image, you finally get 19 pairs of outputs. After it, you compute LPIPS distance between 19 pairs of outputs and take the average. In your experiment, you do the same operation on 100 inputs and get 1900 pairs of outputs in total, which are used to compute average LPIPS distance. Is it right? @richzhang

[richzhang]

Yup! @WorkingCC

[WorkingCC]

@richzhang ,thanks a lot for your patience.

[ShihuaHuang95]

Hi,
Thanks for your great work. For the LPIPS metric, I have one problem, you had tested the LPIPS distance on real images in your corresponding paper, however, as far as I know, there is no paired images in real image set, how can you get that score? Does it mean you compute the distance between random selected real images (not pair, but same domain)?

[richzhang]

Yes, they are randomly selected real images. It serves as a "ceiling" -- the results that an algorithm generates given a single A should not be greater than the variation given random ground truth images B.

[ShihuaHuang95]

Yes, I see. Thank you for your patience.

[comzzw]

For each input image, you sample two random codes Z and generated two random outputs at one time. And you do it up to 19 times, i.e., for each input image, you finally get 19 pairs of outputs. After it, you compute LPIPS distance between 19 pairs of outputs and take the average. In your experiment, you do the same operation on 100 inputs and get 1900 pairs of outputs in total, which are used to compute average LPIPS distance. Is it right? @richzhang

I think this guy may have some wrong understandings. The right way the collaborator explains is that first you sample 20 images for each real input image, and then you average the distances over all possible pairs of these 20 images which is C_20^2=1900. Finally, this process is repeated over all test images and you get the final score which means if you have n test images, there are 1900 \times n pairs are averaged.