Lower mAP

[fishmarch]

Hi,

When I evaluate with your provided maptr_tiny_r50_24e.pth, it seems get a obviously lower mAP than that in your log. Is this reasonable, or maybe I set something incorrect？

-*-*-*-*-*-*-*-*-*-*use metric:chamfer-*-*-*-*-*-*-*-*-*-*
-*-*-*-*-*-*-*-*-*-*threshhold:0.5-*-*-*-*-*-*-*-*-*-*
cls:divider done in 10.783781s!!
cls:ped_crossing done in 2.050395s!!
cls:boundary done in 5.882559s!!

+--------------+-------+--------+--------+-------+
| class        | gts   | dets   | recall | ap    |
+--------------+-------+--------+--------+-------+
| divider      | 27332 | 125430 | 0.472  | 0.266 |
| ped_crossing | 6406  | 45757  | 0.233  | 0.088 |
| boundary     | 21050 | 129763 | 0.341  | 0.146 |
+--------------+-------+--------+--------+-------+
| mAP          |       |        |        | 0.167 |
+--------------+-------+--------+--------+-------+
-*-*-*-*-*-*-*-*-*-*threshhold:1.0-*-*-*-*-*-*-*-*-*-*
cls:divider done in 11.069366s!!
cls:ped_crossing done in 2.089410s!!
cls:boundary done in 5.985780s!!

+--------------+-------+--------+--------+-------+
| class        | gts   | dets   | recall | ap    |
+--------------+-------+--------+--------+-------+
| divider      | 27332 | 125430 | 0.741  | 0.543 |
| ped_crossing | 6406  | 45757  | 0.579  | 0.391 |
| boundary     | 21050 | 129763 | 0.684  | 0.503 |
+--------------+-------+--------+--------+-------+
| mAP          |       |        |        | 0.479 |
+--------------+-------+--------+--------+-------+
-*-*-*-*-*-*-*-*-*-*threshhold:1.5-*-*-*-*-*-*-*-*-*-*
cls:divider done in 10.957865s!!
cls:ped_crossing done in 2.032547s!!
cls:boundary done in 5.931236s!!

+--------------+-------+--------+--------+-------+
| class        | gts   | dets   | recall | ap    |
+--------------+-------+--------+--------+-------+
| divider      | 27332 | 125430 | 0.848  | 0.673 |
| ped_crossing | 6406  | 45757  | 0.778  | 0.609 |
| boundary     | 21050 | 129763 | 0.839  | 0.697 |
+--------------+-------+--------+--------+-------+
| mAP          |       |        |        | 0.660 |
+--------------+-------+--------+--------+-------+
divider: 0.49397581815719604
ped_crossing: 0.3623682012160619
boundary: 0.44848064581553143
map: 0.4349415550629298
{'NuscMap_chamfer/divider_AP': 0.49397581815719604, 
'NuscMap_chamfer/ped_crossing_AP': 0.3623682012160619, 
'NuscMap_chamfer/boundary_AP': 0.44848064581553143, 
'NuscMap_chamfer/mAP': 0.4349415550629298,
'NuscMap_chamfer/divider_AP_thr_0.5': 0.2660278081893921, 
'NuscMap_chamfer/divider_AP_thr_1.0': 0.5427717566490173, 
'NuscMap_chamfer/divider_AP_thr_1.5': 0.6731278896331787,
'NuscMap_chamfer/ped_crossing_AP_thr_0.5': 0.08782391250133514, 
'NuscMap_chamfer/ped_crossing_AP_thr_1.0': 0.3905932307243347, 
'NuscMap_chamfer/ped_crossing_AP_thr_1.5': 0.6086874604225159, 
'NuscMap_chamfer/boundary_AP_thr_0.5': 0.1457921862602234,  
'NuscMap_chamfer/boundary_AP_thr_1.0': 0.5028878450393677, 
'NuscMap_chamfer/boundary_AP_thr_1.5': 0.6967619061470032}

[LegendBC]

If you prepare the environment as we detailed in the doc and evluate the model we provide, the result should be the same. Can you provide your detailed environment(package and version)

[LegendBC]

If you prepare the environment as we detailed in the doc and evluate the model we provide, the result should be the same. Can you provide your detailed environment(package and version)

[fishmarch]

Yes. When changing to a new environment, I get the desired results, although I don't know which package is wrong yet. I will have a check. Thanks.

[alfredgu001324]

@fishmarch Hi, I am having a similar problem here. Did you figure out which package went wrong?

[jasmine-97]

hello， could you tell me how to evaluate the results and output the map , which parameter or config should i correct? Wish to your reply! Thank you!