Question about AP in Validation set

[chrisjuniorli]

Hi

Firstly, much thanks for your code release.

I successfully trained the avod-cars network to 120000 iteration, however, when i run the evaluation command:

python avod/experiments/run_evaluation.py --pipeline_config=avod/configs/pymarid_cars_with_aug_example.config --device='0' --data_split='val'

the result on 120000 iteration is as follows:

120000
done.
car_detection AP: 22.552376 24.332737 25.962851
car_detection_BEV AP: 22.371897 23.603966 25.545847
car_heading_BEV AP: 22.340508 23.495865 25.304886
car_detection_3D AP: 21.757717 19.721174 20.458136
car_heading_3D AP: 21.725494 19.660765 20.347580

which is much lower than the results in the paper(more than 70% basically). And I also run the evaluation on the checkpoint 110000 iteration, which is (26.233753 23.378477 27.482744) on car_detection_3D AP performance.

[chrisjuniorli]

by the way, I change the checkpoint interval from 1000 to 10000, the evaluations of checkpoint from 10000 to 120000 on car_detection_3D AP performance are basically less than 30%(moderate).

[asharakeh]

Cant know the problem from the information you gave. Other people have independently tested the architecture and gotten 74% on validation set.

[chrisjuniorli]

@asharakeh Do I need to change configuration in the kitti_native_eval folder considering that I only trained the avod net on cars? Thanks for your reply.

[chrisjuniorli]

@melfm @kujason can you help me with this problem, I am really confused of the weird results.

[WuYiYiYi]

@chrisjuniorli
Do you solve this problem?I have a question that kitti_native_eval ｈｏｗ to run and how to run this ap ?Can you help me?Thanks a lot!

[kujason]

Something is likely wrong with your dataset, or something has been modified from the recommended setup steps in the readme. Please try re-downloading the dataset and follow the same installation paths as recommended in the readme during setup.