Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please, how can I calculate the MAP for these results? #7754

Open
abderrazzak555 opened this issue Nov 4, 2019 · 3 comments
Open

Please, how can I calculate the MAP for these results? #7754

abderrazzak555 opened this issue Nov 4, 2019 · 3 comments

Comments

@abderrazzak555
Copy link

INFO:tensorflow:Restoring parameters from test_image1/model.ckpt-50000
INFO:tensorflow:Restoring parameters from test_image1/model.ckpt-50000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Performing evaluation on 4 images.
INFO:tensorflow:Performing evaluation on 4 images.
creating index...
index created!
INFO:tensorflow:Loading and preparing annotation results...
INFO:tensorflow:Loading and preparing annotation results...
INFO:tensorflow:DONE (t=0.00s)
INFO:tensorflow:DONE (t=0.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE (t=0.06s).
Accumulating evaluation results...
DONE (t=0.01s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.211
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.380
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.187
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.003
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.233
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.485
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.067
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.241
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.261
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.020
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.283
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.570
INFO:tensorflow:Finished evaluation at 2019-10-28-14:06:58
INFO:tensorflow:Finished evaluation at 2019-10-28-14:06:58
INFO:tensorflow:Saving dict for global step 50000: DetectionBoxes_Precision/mAP = 0.21072435, DetectionBoxes_Precision/mAP (large) = 0.485231, DetectionBoxes_Precision/mAP (medium) = 0.23299623, DetectionBoxes_Precision/mAP (small) = 0.0032204273, DetectionBoxes_Precision/mAP@.50IOU = 0.38036388, DetectionBoxes_Precision/mAP@.75IOU = 0.1866721, DetectionBoxes_Recall/AR@1 = 0.06658986, DetectionBoxes_Recall/AR@10 = 0.24055299, DetectionBoxes_Recall/AR@100 = 0.26059908, DetectionBoxes_Recall/AR@100 (large) = 0.57045454, DetectionBoxes_Recall/AR@100 (medium) = 0.2825, DetectionBoxes_Recall/AR@100 (small) = 0.02, Loss/BoxClassifierLoss/classification_loss = 0.5046802, Loss/BoxClassifierLoss/localization_loss = 0.3415953, Loss/RPNLoss/localization_loss = 0.54075974, Loss/RPNLoss/objectness_loss = 0.46926486, Loss/total_loss = 1.8563001, global_step = 50000, learning_rate = 0.0002, loss = 1.8563001
INFO:tensorflow:Saving dict for global step 50000: DetectionBoxes_Precision/mAP = 0.21072435, DetectionBoxes_Precision/mAP (large) = 0.485231, DetectionBoxes_Precision/mAP (medium) = 0.23299623, DetectionBoxes_Precision/mAP (small) = 0.0032204273, DetectionBoxes_Precision/mAP@.50IOU = 0.38036388, DetectionBoxes_Precision/mAP@.75IOU = 0.1866721, DetectionBoxes_Recall/AR@1 = 0.06658986, DetectionBoxes_Recall/AR@10 = 0.24055299, DetectionBoxes_Recall/AR@100 = 0.26059908, DetectionBoxes_Recall/AR@100 (large) = 0.57045454, DetectionBoxes_Recall/AR@100 (medium) = 0.2825, DetectionBoxes_Recall/AR@100 (small) = 0.02, Loss/BoxClassifierLoss/classification_loss = 0.5046802, Loss/BoxClassifierLoss/localization_loss = 0.3415953, Loss/RPNLoss/localization_loss = 0.54075974, Loss/RPNLoss/objectness_loss = 0.46926486, Loss/total_loss = 1.8563001, global_step = 50000, learning_rate = 0.0002, loss = 1.8563001
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 50000: test_image1/model.ckpt-50000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 50000: test_image1/model.ckpt-50000
INFO:tensorflow:Performing the final export in the end of training.
INFO:tensorflow:Performing the final export in the end of training.

@tensorflowbutler tensorflowbutler added the stat:awaiting response Waiting on input from the contributor label Nov 5, 2019
@tensorflowbutler
Copy link
Member

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
What is the top-level directory of the model you are using
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

@abderrazzak555
Copy link
Author

System information
What is the top-level directory of the model you are using:
models/research/object_detection/
Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Linux Ubuntu 18.04
TensorFlow installed from (source or binary):
conda install
TensorFlow version (use command below):
1.12.0
Bazel version (if compiling from source):
CUDA/cuDNN version:
CUDA® Toolkit 9.0; cuDNN v7.0
GPU model and memory:
GForce GTX 1080Ti (11GB)

python eval.py \ --logtostderr \ --pipeline_config_path=training/faster_rcnn_inception_v2_pets.config\ --checkpoint_dir=training/ \ --eval_dir=eval/

@tensorflowbutler tensorflowbutler removed the stat:awaiting response Waiting on input from the contributor label Nov 6, 2019
@asr-aditya
Copy link

@abderrazzak555 can you share the config file for these results?

@ravikyram ravikyram added the models:research models that come under research directory label Jun 19, 2020
@jaeyounkim jaeyounkim added models:research:odapi ODAPI and removed models:research models that come under research directory labels Jun 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants