Skip to content
This repository has been archived by the owner on Dec 21, 2023. It is now read-only.

Numerical differences in OD evaluation (in 5.8 and 6.0) on CPU and GPU #2865

Closed
srikris opened this issue Dec 17, 2019 · 0 comments · Fixed by #2955
Closed

Numerical differences in OD evaluation (in 5.8 and 6.0) on CPU and GPU #2865

srikris opened this issue Dec 17, 2019 · 0 comments · Fixed by #2955
Assignees
Labels
Milestone

Comments

@srikris
Copy link
Contributor

srikris commented Dec 17, 2019

For evaluation of an object detector, there are some slight variance between the

In version 5.8, we have

## MPS
Out[4]:
{'average_precision_50': {'Bagel': 0.8520384669200912,
  'Banana': 0.8575615741065546,
  'Coffee': 0.5958508158508159,
  'Croissant': 0.7764094658504597,
  'Egg': 0.6904679838895471,
  'Waffle': 0.719937989567849},
 'mean_average_precision_50': 0.7487110493642196}

## MXNET
Out[6]:
{'average_precision_50': {'Bagel': 0.852246094152459,
  'Banana': 0.8586351107478006,
  'Coffee': 0.5961538461538463,
  'Croissant': 0.7625511019920956,
  'Egg': 0.6781562921839436,
  'Waffle': 0.6855548346869123},
 'mean_average_precision_50': 0.7388828799861763}

and in version 6.0, we have

## MPS
{'average_precision_50': {'Coffee': 0.5881511569023132, 
  'Croissant': 0.7716814875602722, 
  'Waffle': 0.6675131916999817, 
  'Bagel': 0.8474096059799194, 
  'Egg': 0.6781503558158875, 
  'Banana': 0.8502914309501648}, 
'mean_average_precision_50': 0.7338662147521973}

## TF
{'average_precision_50': {'Coffee': 0.5817939043045044, 
  'Croissant': 0.7605029940605164, 
  'Waffle': 0.6415750980377197, 
  'Bagel': 0.8460806012153625, 
  'Egg': 0.6689953804016113, 
  'Banana': 0.8549931645393372}, 
'mean_average_precision_50': 0.7256568074226379}
@srikris srikris added the p2 label Dec 17, 2019
@srikris srikris added this to the 6.1 milestone Dec 17, 2019
@srikris srikris changed the title Evaluation of object detector (in 5.8 and 6.0) is slightly different on CPU and GPU OD evaluation (in 5.8 and 6.0) is slightly different on CPU and GPU Dec 17, 2019
@srikris srikris changed the title OD evaluation (in 5.8 and 6.0) is slightly different on CPU and GPU Numerical differences in OD evaluation (in 5.8 and 6.0) on CPU and GPU Dec 17, 2019
@jakesabathia2 jakesabathia2 self-assigned this Jan 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants