COCO evaluation results do not match

[samiarshad]

Hi,

Thanks for sharing your work. I have executed the validation code from here on COCO validation set and received the results below. Could you please confirm if this is the expected output?

Thanks

------------------------------------------------------------- COCO Evaluation Results -----------------------------------------------
loading annotations into memory...
Done (t=1.38s)
creating index...
index created!
Loading and preparing results...
DONE (t=7.31s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE (t=51.15s).
Accumulating evaluation results...
DONE (t=11.30s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.002
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.006
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.007
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.008
Running per image evaluation...
Evaluate annotation type segm
DONE (t=60.37s).
Accumulating evaluation results...
DONE (t=10.75s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.002
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.005
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.006
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.006

[berry-ding]

hi, could you provide the json file

[samiarshad]

Hello,

Please find the JSON file attached.

Thanks

predictions.json

[MMa321]

Hello, thank you for your sharing. I also want to evaluate this model on the coco dataset. May I ask how did you deploy it and how could you get the result

[Armadillooutfitter]

Dear MMa321, I deployed the model using Docker and ran it on the COCO dataset. I obtained the results by running the evaluation script provided with the model. Best regards, Armadillo Outfitter

…

On Sun, Feb 25, 2024 at 11:01 PM MMa321 ***@***.***> wrote: Hello, thank you for your sharing. I also want to evaluate this model on the coco dataset. May I ask how did you deploy it and how could you get the result — Reply to this email directly, view it on GitHub <#183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BA3Y3ZYWOL7YYC6RTSFCCBLYVQCIBAVCNFSM6AAAAAA7AHQ46SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGI3TOMZYGY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[MMa321]

尊敬的MMa321， 我使用 Docker 部署了模型，并在 COCO 数据集上运行了它。我 通过运行随 型。 此致敬意 犰狳装备
...
2024 年 2 月 25 日星期日晚上 11：01 MMa321 @.>写道： 您好，谢谢您的分享。我还想评估这个模型 Coco 数据集。请问您是如何部署它的，以及如何获得 结果 — 直接回复此邮件，在 GitHub 上查看 <#183（评论）>， 或取消订阅 https://github.com/notifications/unsubscribe-auth/BA3Y3ZYWOL7YYC6RTSFCCBLYVQCIBAVCNFSM6AAAAAA7AHQ46SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGI3TOMZYGY . 您收到此消息是因为您订阅了此线程。消息 ID： @.>

Hello.
Thank you for your reply.
Could you please provide the specific code？

[MMa321]

晉語

感谢您分享您的作品。我已经在COCO验证集上执行了验证代码，并收到了以下结果。您能否确认这是否是预期的输出？

谢谢

------------------------------------------------------------- COCO 评估结果-----------------------------------------------将注释加载到内存中...完成 （t=1.38s） 创建索引...索引已创建！加载和准备结果...完成 （t=7.31s） 正在创建索引...索引已创建！正在按图像评估运行...评估注释类型 bbox DONE （t=51.15s）。累积评估结果...完成 （t=11.30s）。平均精度 （AP） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.000平均精度 （AP） @[ IoU=0.50：0.95 | area= large | maxDets=100 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 1 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 10 ] = 0.002 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.006 平均召回率 （AR） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.004平均召回率 （AR） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.007 平均召回率 （AR） @[ IoU=0.50：0.95 | area= large | maxDets=100 ] = 0.008 每个图像评估正在运行...计算注释类型 segm DONE （t=60.37s）。累积评估结果...完成 （t=10.75s）。平均精度 （AP） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.000平均精度 （AP） @[ IoU=0.50：0.95 | area= 大 | maxDets=100 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 1 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 10 ] = 0.002 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.005 平均召回率 （AR） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.004平均召回率 （AR） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.006 平均召回率 （AR） @[ IoU=0.50：0.95 | area= large | maxDets=100 ] = 0.006

Hello, I always report errors in the verification process, is it convenient for you to share your verification code