A tiny package supporting distributed computation of COCO metrics (like mAP) for PyTorch models.
I made this package available on PyPi (thanks to this guide): https://pypi.org/project/coco-eval/.
pip install coco-eval
The metric was taken and isolated from the DETR repository. Credits go to the authors.
High-level usage is as follows (assuming you have a PyTorch model that makes predictions):
from coco_eval import CocoEvaluator
from torchvision.datasets import CocoDetection
from torch.utils.data import DataLoader
dataset = CocoDetection(root="path_to_your_images", annFile="path_to_annotation_file")
dataloader = DataLoader(dataset, batch_size=2)
evaluator = CocoEvaluator(coco_gt=dataset.coco, iou_types=["bbox"])
model = ...
for batch in dataloader:
predictions = model(batch)
evaluator.update(predictions)
evaluator.synchronize_between_processes()
evaluator.accumulate()
evaluator.summarize()
Refer to my DETR fine-tuning demo notebook regarding an example of using it.