Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to get the consistency measurement? #2

Open
kigane opened this issue Oct 18, 2022 · 5 comments
Open

How to get the consistency measurement? #2

kigane opened this issue Oct 18, 2022 · 5 comments

Comments

@kigane
Copy link

kigane commented Oct 18, 2022

E(Oi, Oj) = LPIPS(Oi, Mi,j, Wi,j(Oj)), how to get the mask and how to apply it to lpips?

@semchan
Copy link
Owner

semchan commented Oct 19, 2022

E(Oi, Oj) = LPIPS(Oi, Mi,j, Wi,j(Oj)), how to get the mask and how to apply it to lpips?

We totally followed the link (https://github.com/phoenix104104/fast_blind_video_consistency) to calculate lpips, please find the detail in there. thanks

@kigane
Copy link
Author

kigane commented Oct 20, 2022

I have read their code. in evaluate_LPIPS.py they use LPIPS to get the perceptual distance between processed image P and their model output O. But, P and O are the same frame of the video. in evaludate_WarpError.py, they use optical flow predicted by FlowNet2 betweent frame1 and frame2 to warp frame2 to frame1, then calculate the L2 distance on non-occlude pixels. They do not use masks on LPIPS metric.
As far as I know, LPIPS use vgg/squeeze/alex net to extract feature maps of differen layers of two input images, then calculate the L2 distance. So I am really confused about the mask Mi,j used in the equation. Could you please explain this detail more clearly? thank you.

@koolo233
Copy link

@kigane Have you solved this problem? It strange that none of StylizedNeRF, StyleRF, Learning to Stylize Novel Views, etc. provide a calculation method of consistency.

@zAuk000
Copy link

zAuk000 commented Jun 7, 2023

@kigane Have you solved this problem? It strange that none of StylizedNeRF, StyleRF, Learning to Stylize Novel Views, etc. provide a calculation method of consistency.

I have the same doubt as well. Why hasn't the calculation method for quantitative indicators been provided, even though it's the only evaluation criterion?

@zAuk000
Copy link

zAuk000 commented Jun 7, 2023

I have read their code. in evaluate_LPIPS.py they use LPIPS to get the perceptual distance between processed image P and their model output O. But, P and O are the same frame of the video. in evaludate_WarpError.py, they use optical flow predicted by FlowNet2 betweent frame1 and frame2 to warp frame2 to frame1, then calculate the L2 distance on non-occlude pixels. They do not use masks on LPIPS metric. As far as I know, LPIPS use vgg/squeeze/alex net to extract feature maps of differen layers of two input images, then calculate the L2 distance. So I am really confused about the mask Mi,j used in the equation. Could you please explain this detail more clearly? thank you.

Have you tried testing the generated results using the code from "warperror.py"? If so, are the results close to those in the paper?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants