How to calculate R-precision?

[priyankaupadhyay090]

@wtliao

I wanted to calculate R-precision for comparisons with few state-of-the-art benchmarks so I adjusted the code into main.py to calculate R-precision using already given finetuned trained model by you. I achieved 73% value around. I was just checking your paper newest (5th) version where you have reported R precision 86%. Can you please share your code to calculate the R-precision so that I can figure out why we have large differences in our R precision scores.

Thank you

[wtliao]

@priyankaupadhyay090
we test the R-precision with the instruction of AttGan (https://arxiv.org/abs/1711.10485) with the code https://github.com/MinfengZhu/DM-GAN/blob/master/code/trainer.py in lines 469-493
We are doing extension research base on this work. So we won't update this repo until the research is done.

BWT: I cannot approach the email you sent me for asking questions because I am not in the institute anymore.

[wtliao]

@priyankaupadhyay090 BWT： could you report me your IS and FID scores on CUB and coco? Thanks!

[priyankaupadhyay090]

@priyankaupadhyay090 we test the R-precision with the instruction of AttGan (https://arxiv.org/abs/1711.10485) with the code https://github.com/MinfengZhu/DM-GAN/blob/master/code/trainer.py in lines 469-493 We are doing extension research base on this work. So we won't update this repo until the research is done.

BWT: I cannot approach the email you sent me for asking questions because I am not in the institute anymore.

@wtliao
I followed the same approach to calcualte R precision.
Took instruction from AtnGAN and code from DM-GAN lines 467 -493 (https://github.com/MinfengZhu/DM-GAN/blob/master/code/trainer.py).

I am using CUB dataset and scores I got:
IS = 5.13 ± 0.06
FID = 15.60
R-precision = 73.55 ± 0.68

so I have almost got the same IS and FID which you reported but there is large difference in R-precision score.

[priyankaupadhyay090]

@wtliao

Would it be possible for you to share trained model from which you calculate R-precision ?

It would really help me to figure out that is it previous trained model (provided on the link) or something else which is leading me to get different R-precision score than your score.

[wtliao]

@priyankaupadhyay090
By recomputing the R-precision using the provided pre-trained model on CUB our result is 75.85 ± 0.92. Your results are correct.