RF-DETR 1.8.0: Keypoint Detection

📋 Summary

RF-DETR 1.8.0 introduces RFDETRKeypointPreview — a keypoint detection model that fits naturally into the same .train() / .predict() workflow you already know. It ships with COCO keypoint AP evaluation out of the box, and its predictions include per-keypoint uncertainty estimates alongside coordinates and visibility scores. This release also fixes loss scaling for gradient accumulation in keypoint training, adds export_for_roboflow for creating offline upload bundles, and ships five checkpoint and device-detection reliability fixes. The long-deprecated simplify and force kwargs are removed from RFDETR.export.

✨ Spotlights

🎯 Keypoint detection — RFDETRKeypointPreview

RFDETRKeypointPreview brings keypoint detection to RF-DETR with the same interface you use for object detection and segmentation. It produces sv.KeyPoints predictions carrying per-detection keypoint coordinates, visibility scores, and covariance matrices you can use to reason about prediction uncertainty.

The Preview in the name signals this is an early-access capability — the API and model behavior are stable enough for experimentation and real projects, but expect continued iteration based on community feedback before it graduates to a non-preview release.

from rfdetr import RFDETRKeypointPreview
from rfdetr.config import KeypointTrainConfig
from rfdetr.datasets._keypoint_schema import infer_coco_keypoint_schema

schema = infer_coco_keypoint_schema("dataset/train/_annotations.coco.json")

model = RFDETRKeypointPreview()
model.train(
    dataset_dir="dataset/",
    config=KeypointTrainConfig(epochs=50, batch_size=8, keypoint_schema=schema),
)

results = model.predict("image.jpg")
print(results.xy)  # (N, K, 2) keypoint coordinates
print(results.keypoint_confidence)  # (N, K) per-keypoint visibility

To visualize prediction uncertainty as pixel-space ellipses, convert the stored covariance matrices:

from rfdetr.utilities.keypoints import precision_cholesky_to_pixel_covariance

cov = precision_cholesky_to_pixel_covariance(
    results.data["covariance"],
    source_shape=image.shape[:2],
)  # (N, K, 2, 2)

New public APIs: KeypointTrainConfig, RFDETRKeypointPreviewConfig from rfdetr.config; precision_cholesky_to_pixel_covariance from rfdetr.utilities; schema helpers infer_coco_keypoint_schema, CocoKeypointSchema, active_keypoint_counts from rfdetr.datasets._keypoint_schema. There's also a fine-tuning cookbook (docs/cookbooks/fine-tune_keypoints.ipynb) with an end-to-end walkthrough covering dataset setup, schema inference, training, and inference with uncertainty.

🔧 Gradient accumulation fix for keypoint training

If you train keypoint models with accumulate_grad_batches > 1, losses were previously normalized per mini-batch instead of across the full effective batch. This is now fixed automatically — no code changes needed.

# Losses now correctly normalized over the full effective batch
# (batch_size × accumulate_grad_batches)
model.train(
    dataset_dir="dataset/",
    config=KeypointTrainConfig(batch_size=4, accumulate_grad_batches=4),
)

Object detection and segmentation models are not affected.

📦 Local Roboflow bundle export — RFDETR.export_for_roboflow

Creates a self-contained Roboflow upload bundle (weights.pt + class_names.txt) on disk without making any network calls. Useful for air-gapped environments or reviewing the bundle before uploading.

model.export_for_roboflow("roboflow_upload/")
# → roboflow_upload/weights.pt
# → roboflow_upload/class_names.txt

Existing deploy_to_roboflow() calls work unchanged — they now delegate to this method internally.

⚠️ Migration guide

Full guide: rfdetr.roboflow.com/latest/getting-started/migration/

❌ Remove deperacted simplify and force from RFDETR.export calls

Both kwargs were deprecated in v1.6.0 and are removed in v1.8.0. They were no-ops throughout their deprecation period.

# Before (DeprecationWarning since v1.6.0):
model.export(output_dir="out/", simplify=True, force=True)

# After:
model.export(output_dir="out/")

📝 rfdetr.datasets.aug_config → rfdetr.datasets.aug_configs

The module name changed from singular to plural. If you import from it directly:

# Before:
from rfdetr.datasets.aug_config import AUG_AGGRESSIVE

# After:
from rfdetr.datasets.aug_configs import AUG_AGGRESSIVE

All preset constants (AUG_AGGRESSIVE, AUG_MOSAIC, etc.) are unchanged.

🔗 Dependency updates

• supervision>=0.29.0 is now required (for sv.KeyPoints support).
• pyDeprecate constraint narrowed to >=0.9,<0.10 (was >=0.6,<0.8). Update your environment if you're pinned to the older range.

📝 Notable changes

🚀 Added

• RFDETRKeypointPreview — keypoint detection with covariance-based uncertainty and COCO keypoint AP evaluation. (#1099)
• MetricKeypointOKS — reusable OKS metric exported from rfdetr.evaluation. Supports arbitrary keypoint counts, per-category sigmas, and multi-GPU evaluation. (#1107)
• RFDETR.export_for_roboflow(output_dir) — local Roboflow bundle without a network call; deploy_to_roboflow delegates to this. (#1086)
• Keypoint fine-tuning cookbook (docs/cookbooks/fine-tune_keypoints.ipynb) — end-to-end walkthrough: dataset download, schema inference, training, and inference with uncertainty. (#1104)

🌱 Changed

• DDP strategy — find_unused_parameters=True is now set unconditionally for all model variants under strategy='ddp' or 'auto'. Previously only segmentation set this flag. To opt out: trainer_kwargs={"strategy": DDPStrategy(find_unused_parameters=False)}. (#1094)
• rfdetr.datasets.aug_config renamed to rfdetr.datasets.aug_configs — update any direct imports; all constants are unchanged. (#1103)

🗑️ Removed

• RFDETR.export(simplify=..., force=...) — removed after deprecation since v1.6.0; both were no-ops. (#1102)

🔧 Fixed

• Gradient accumulation for keypoint training — loss scaling now correct across the accumulated effective batch. (#1117)
• from_checkpoint() fine-tuning on a different class count — checkpoint-derived class count no longer overrides user intent; fine-tuning on a different class count now adapts the head correctly. (#1106)
• Explicit num_classes honored when it equals the default — passing num_classes=N where N matches the model default was previously ignored; the explicit value is now always respected. (#1109)
• from_checkpoint() with starter checkpoints — model variant is now inferred from the checkpoint filename when pretrain_weights is absent or unset. (#1065)
• Device auto-detection — accelerator availability is now verified before selecting a device; prevents assigning CUDA on machines that have the CUDA headers installed but no driver. (#1111)
• Spurious warning on keypoint datasets — a false mismatch warning triggered on custom datasets when auto-adjusting num_classes beyond the schema length is now suppressed. (#1113)
• Scale jitter restored in non-square training crop. (#1088)
• Multi-GPU validation deadlock in COCO mAP sync across zero-batch ranks. (#1085)
• import rfdetr no longer fails on NumPy 2.x alongside dependencies that reference np.complex_. (#1064)
• rfdetr_plus availability check corrected — false-positive when the package is partially installed. (#1083)

---

🏆 Contributors

• Jirka Borovec (@Borda, LinkedIn) — keypoint detection pipeline, gradient accumulation fix, OKS metric, release
• Anatoly Ryabchenko (@anatoly-ryabchenko, LinkedIn) — checkpoint reliability improvements (class count handling, device detection)
• Aaryan Kangte (@Aaryan562, LinkedIn) — multi-GPU validation deadlock fix
• Dohyeon Yoon (@dohyeonYoon, LinkedIn) — DDP find_unused_parameters fix
• Lee Clement (@leeclemnet, LinkedIn) — export_for_roboflow extraction

---

Full changelog: 1.7.0...1.8.0