Questions about LM-O evaluation results

[Liuchongpei]

Hi! thanks for your great work.
I execute the following command to get the evaluation results of LM-O as follows, ‘GDR-Net-DATA‘ is the folder where I put the trained models.
./core/gdrn_modeling/test_gdrn.sh configs/gdrn/lmo/a6_cPnP_AugAAETrunc_BG0.5_lmo_real_pbr0.1_40e.py 1 GDR-Net-DATA/gdrn/lmo/a6_cPnP_AugAAETrunc_BG0.5_lmo_real_pbr0.1_40e/gdrn_lmo_real_pbr.pth

Is ‘ad_10’ the ‘Average Recall (%) of ADD(-S)’ mentioned in Table 2 in the paper?

[wangg12]

yes

[Liuchongpei]

There is a big gap between the results of my evaluation and the results in the paper. Did I make a mistake?

[wangg12]

The result is correct.

yours: 56.19
paper: 56.1

[Liuchongpei]

Thank you, my fault, I thought it corresponds to the column with ‘P.E.’ as N.

[Liuchongpei]

Sorry to disturb you again! This time it is about the evaluation results of YCB-V.
I execute the following command to get the evaluation results of YCB-V.
./core/gdrn_modeling/test_gdrn.sh configs/gdrn/ycbv/a6_cPnP_AugAAETrunc_BG0.5_Rsym_ycbv_real_pbr_visib20_10e.py 0 GDR-Net-DATA/gdrn/ycbv/a6_cPnP_AugAAETrunc_BG0.5_Rsym_ycbv_real_pbr_visib20_10e/gdrn_ycbv.pth

I guess the correspondence is like this:

 in the picture  | in the paper
 ad_0.100: 39.00   ADD(-S): 49.1  
 AUCad_10: 72.88   AUC of ADD(-S): 80.2
 AUCadi_10: 86.50  AUC of ADD-S: 89.1

There seems to be a gap here, I am puzzled.

[wangg12]

I have re-tested it. No gap at all.

[Liuchongpei]

I tried it again, and the gap disappeared. It is too wierd.