v1.8.2: YOLO Pose Support, Active-First Keypoints

📋 Summary

RF-DETR 1.8.2 rounds out the keypoint detection feature set with YOLO pose dataset support (load Ultralytics YOLO pose datasets directly for training, no conversion needed), an active-first keypoint schema default that makes class IDs zero-based by default, and a new amp_dtype field for explicit fp16/bf16 mixed-precision control. Two new cookbooks land: instance segmentation fine-tuning and an inference latency benchmark. On the reliability side, a long-standing bug in from_checkpoint() that silently inflated num_classes by one is fixed, TensorRT ONNX export is unblocked for all model variants, and several inference correctness issues are resolved. Keypoint users: the default schema changed from background-first [0, 17] to active-first [17]. Checkpoint weights load unchanged; class IDs in inference output shift — see the migration guide below.

✨ Spotlights

YOLO pose keypoint datasets

Train keypoint models directly from Ultralytics YOLO pose datasets. Point dataset_dir at any dataset folder with a data.yaml containing kpt_shape — schema, keypoint names, and OKS sigmas are inferred automatically.

from rfdetr import RFDETRKeypointPreview
from rfdetr.config import KeypointTrainConfig

model = RFDETRKeypointPreview()
model.train(
    KeypointTrainConfig(
        dataset_dir="path/to/yolo-pose-dataset",
        epochs=50,
    )
)

Active-first keypoint schema — cleaner class IDs

Person is now at class_id=0 instead of class_id=1. Legacy checkpoints load without any changes — RF-DETR auto-detects the schema at load time. New schema utilities make conversion explicit when needed:

from rfdetr.utilities.keypoints import _is_bg_first_schema, _to_active_first

if _is_bg_first_schema(schema):
    schema = _to_active_first(schema)  # [0, 17] → [17]

amp_dtype on TrainConfig — pin fp16 or bf16

Stop relying on device auto-detection. Set the AMP dtype explicitly:

from rfdetr.config import TrainConfig

config = TrainConfig(dataset_dir="...", amp_dtype="fp16")  # force fp16
config = TrainConfig(dataset_dir="...", amp_dtype="bf16")  # force bf16
config = TrainConfig(dataset_dir="...", amp_dtype="auto")  # default, device heuristic

TensorRT ONNX export — now works

spatial_shapes in Transformer.forward() is now built from symbolic Shape ops, removing the ScatterND node that TensorRT rejected with "IScatterLayer cannot be used to compute a shape tensor". All RF-DETR variants can now export to TensorRT engines:

# After export(format="onnx"):
trtexec --onnx=model.onnx --saveEngine=model.engine

New cookbooks

Two new end-to-end notebooks:

• Instance segmentation fine-tuning (docs/cookbooks/fine-tune_segmentation.ipynb) — RFDETRSegSmall across seven diverse segmentation datasets with training metrics and sample previews.
• Inference latency benchmark (docs/cookbooks/inference-latency-benchmark.ipynb) — reproducible CPU/GPU throughput measurements across model sizes.

🔄 Migration guide

🌱 Changed: keypoint class IDs shift to zero-based — checkpoint weights unaffected

Affects RFDETRKeypointPreview / RFDETRKeypointPreviewConfig users.

Checkpoint weights load unchanged — RF-DETR auto-detects the schema from the checkpoint and aligns it at load time. No re-training or weight migration needed.

What breaks: class IDs in inference output shift. Person moves from class_id=1 to class_id=0. Post-processing code that hardcodes class IDs must update:

# Before (background-first [0, 17]: person was at class_id=1)
class_name = "person" if detection.class_id == 1 else "other"

# After (active-first [17]: person is at class_id=0)
class_name = "person" if detection.class_id == 0 else "other"

Schema-agnostic alternative (works with either schema):

class_name = detection.data["class_name"]

To keep the legacy schema, pass num_keypoints_per_class at construction time:

config = RFDETRKeypointPreviewConfig(num_keypoints_per_class=[0, 17])

📝 Notable changes

🚀 Added

• YOLO pose keypoint dataset support — load Ultralytics YOLO pose datasets (.yaml with kpt_shape) directly for keypoint training. Schema inferred via infer_yolo_keypoint_schema. (#1156)
• amp_dtype on TrainConfig — pin mixed-precision dtype to "auto" / "bf16" / "fp16". Invalid values degrade to "auto" with a UserWarning. (#1143)
• Keypoint schema utilities — is_bg_first_schema, to_active_first, to_bg_first, schemas_semantically_equal in rfdetr.utilities.keypoints (re-exported from rfdetr.utilities). (#1160)
• Instance segmentation fine-tuning cookbook (docs/cookbooks/fine-tune_segmentation.ipynb). (#1159)
• Inference latency benchmark cookbook (docs/cookbooks/inference-latency-benchmark.ipynb). (#1152)

🌱 Changed

• Default num_keypoints_per_class changed from [0, 17] to [17] in RFDETRKeypointPreviewConfig. Checkpoint weights load unchanged — RF-DETR auto-aligns the schema at load time. Class IDs in inference output shift (person moves from class_id=1 to class_id=0); post-processing code that hardcodes class IDs must update. (#1160)

🔧 Fixed

• from_checkpoint() now reads the correct num_classes — was class_embed.weight.shape[0] (including background), now shape[0] - 1. Prevented shape mismatches and silently added an extra output class to every fine-tuned checkpoint load. BestModelCallback._serialize_model_config also fixed. (#1158)
• TensorRT ONNX export unblocked — spatial_shapes built from symbolic Shape ops, removing the ScatterND that blocked TensorRT compilation. (#1155)
• HungarianMatcher now respects configured focal_alpha — was hardcoded to 0.25, misaligning bipartite matching cost with the actual focal loss. (#1147)
• Keypoint inference class_name corrected — predictions now carry the right class name for keypoint models. (#1151)
• predict() re-asserts eval mode — prevents silent train-mode inference for unoptimized models after the first call. (#1146)
• TFLite inference — preprocessing and mask decoder now match PyTorch predict(). (#1131)
• Python version mismatch in dependency overrides resolved. (#1137)

---

🏆 Contributors

Thanks to everyone who contributed to this release:

• Jirka Borovec (@Borda, LinkedIn) — active-first keypoint schema, YOLO pose support, checkpoint restore fix, segmentation + latency cookbooks
• Anatoly Ryabchenko (@ryabchenko-a) — amp_dtype field for explicit AMP dtype control
• Ruben (@RubenHaisma) — HungarianMatcher focal_alpha fix and predict() eval-mode re-assertion
• Isaac Robinson (@isaacrob, LinkedIn) — TensorRT-safe ONNX export via symbolic Shape ops
• Stefan Schneider (@hinogi) — dataset type refactoring and Python dependency fix
• Omkar Kabde (@omkar-334, LinkedIn) — TFLite inference preprocessing and mask decoder fix

---

Full changelog: 1.8.1...v1.8.2