hotcoco

11-26x faster COCO evaluation — a drop-in replacement for pycocotools that works with Ultralytics YOLO, Detectron2, mmdetection, RF-DETR, and any pycocotools-based pipeline.

Available as a Python package, CLI tool, and Rust library. Pure Rust — no Cython, no C compiler, no Microsoft Build Tools. Prebuilt wheels for Linux, macOS, and Windows.

Documentation | Changelog | Roadmap

Performance

Benchmarked on COCO val2017 (5,000 images, 36,781 ground truth annotations, ~43,700 detections), Apple M1 MacBook Air:

Eval Type	pycocotools	faster-coco-eval	hotcoco
bbox	11.79s	3.47s (3.4x)	0.74s (15.9x)
segm	19.49s	10.52s (1.9x)	1.58s (12.3x)
keypoints	4.79s	3.08s (1.6x)	0.19s (25.0x)

Speedups in parentheses are vs pycocotools. Results verified against pycocotools on COCO val2017 with a 10,000+ case parity test suite — your AP scores won't change.

Quick Start

Python

pip install hotcoco

from hotcoco import COCO, COCOeval

coco_gt = COCO("instances_val2017.json")
coco_dt = coco_gt.load_res("detections.json")

ev = COCOeval(coco_gt, coco_dt, "bbox")
ev.evaluate()
ev.accumulate()
ev.summarize()

Drop-in replacement for pycocotools

If you use Detectron2, Ultralytics YOLO, mmdetection, or any other pycocotools-based pipeline, call init_as_pycocotools() once at startup — no other code changes needed:

from hotcoco import init_as_pycocotools
init_as_pycocotools()

# Existing code works unchanged
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
from pycocotools import mask

LVIS evaluation

hotcoco supports LVIS federated evaluation with all 13 metrics (AP, APr, APc, APf, AR@300, and more). Use LVISeval directly or call init_as_lvis() to drop into any existing lvis-api pipeline:

from hotcoco import COCO, LVISeval

lvis_gt = COCO("lvis_v1_val.json")
lvis_dt = lvis_gt.load_res("detections.json")

ev = LVISeval(lvis_gt, lvis_dt, "segm")
ev.run()
print(ev.get_results())  # {"AP": ..., "APr": ..., "APc": ..., "APf": ..., "AR@300": ...}

# Or as a drop-in for Detectron2 / MMDetection lvis-api pipelines
from hotcoco import init_as_lvis
init_as_lvis()

from lvis import LVIS, LVISEval, LVISResults  # resolves to hotcoco

TIDE error analysis

tide_errors() decomposes every false positive and false negative into six error types — Localization, Classification, Duplicate, Background, Both, and Miss — and reports the ΔAP for each. Use it to understand why your model falls short, not just how much:

ev = COCOeval(coco_gt, coco_dt, "bbox")
ev.evaluate()

result = ev.tide_errors()
for name, delta in sorted(result["delta_ap"].items(), key=lambda x: -x[1]):
    if name not in ("FP", "FN"):
        print(f"  {name}: ΔAP={delta:.4f}  n={result['counts'].get(name, '—')}")

Or from the CLI:

coco eval --gt instances_val2017.json --dt bbox_results.json --tide

CLI

cargo install hotcoco-cli
coco-eval --gt annotations.json --dt detections.json --iou-type bbox

Rust

cargo add hotcoco

use hotcoco::{COCO, COCOeval};
use hotcoco::params::IouType;
use std::path::Path;

let coco_gt = COCO::new(Path::new("annotations.json"))?;
let coco_dt = coco_gt.load_res(Path::new("detections.json"))?;

let mut eval = COCOeval::new(coco_gt, coco_dt, IouType::Bbox);
eval.evaluate();
eval.accumulate();
eval.summarize();

License

MIT