v1.10.1: CUDA Compile Fixes & Memory Reduction

RF-DETR 1.10.1 is a patch release: compile=True now actually reaches CUDA's torch.compile path under the default multi_scale=True recipe, and peak CUDA memory in segmentation loss drops by sampling ground-truth masks one image at a time. It also carries several device-sync and strategy-selection fixes for the still-experimental, undocumented TPU/XLA training path (tracked under issue #1058) — see Notable Changes below. No breaking changes in this release.

✨ Spotlights

⚡ compile=True now actually compiles on CUDA with the default multi_scale=True

The compile gate previously excluded multi-scale training, but the exclusion sat behind checks that already required a CUDA device and CUDA accelerator — so it only ever blocked the CUDA path, where dynamic=True already handles the varying input size. Setting compile=True on a default recipe previously logged a notice and trained eagerly instead. (#1436; #1411, shipped in 1.10.0, made compilation reachable in the first place)

from rfdetr import RFDETRSmall

model = RFDETRSmall()
model.train(dataset_dir="...", compile=True)  # multi_scale=True by default — now actually compiles on CUDA

💾 Reduced peak CUDA memory in segmentation loss

For multi-image CUDA batches with boolean ground-truth masks, sampling now happens one image at a time, releasing each image's float masks before the next, instead of concatenating a batch-wide float mask tensor. Narrower inputs (single-image batches, non-CUDA, non-bool masks) keep the prior concat path unchanged. (#1437)

🔄 Migration guide

No breaking changes in this release. One thing to know: if you call dice_loss_jit / sigmoid_ce_loss_jit directly — not part of the public API, only reachable through rfdetr.models.lwdetr's backward-compat re-exports — most NumPy scalar denominators now need casting to float first. See MIGRATION.md for details.

📝 Notable changes

🔧 Fixed

• compile=True now takes effect on CUDA with the default multi_scale=True, instead of previously logging a notice and training eagerly. Also fixes the now-stale multi_scale=False claim in the COCO2017 training cookbook. (#1436; #1411, shipped in 1.10.0, made compilation reachable in the first place)
• Reduced peak CUDA memory in segmentation loss by sampling matched ground-truth masks one image at a time, for multi-image CUDA batches with boolean masks. (#1437)
• Docs site: the version banner no longer flags the current release as outdated. (#1429)
• TPU/XLA training (experimental, undocumented): point_sample(mode="nearest") no longer leaves the device on MPS/XLA (#1432, issue #1058); SetCriterion.loss_masks no longer syncs its normalizing denominator to the host, with a side effect that dice_loss_jit/sigmoid_ce_loss_jit — reachable only through lwdetr.py's backward-compat re-exports, not the public API — now reject most NumPy scalar denominators (#1428, issue #1058); build_trainer now selects XLAStrategy for multi-device XLA/TPU training when strategy="auto", previously a crash at Trainer construction, and changes the precision-plugin path for single-device accelerator="auto" XLA runs too (#1427, issue #1058); XLA-marked tests pass on real TPU silicon (#1426, issue #1058).

🏆 Contributors

• Jesús Royeth (@JESUSROYETH): all six code/test fixes in this release — the compile=True/CUDA multi-scale fix, reduced peak CUDA memory in segmentation loss, and the experimental TPU/XLA device-sync, strategy-selection, and test-suite fixes.
• Jirka Borovec (@Borda, LinkedIn): fixed the docs-site version banner falsely flagging the current release as outdated, and this release's own prep.

---

Full changelog: 1.10.0...1.10.1

v1.10.0: Faster Training & Inference

🎉 RF-DETR 1.10.0 is a training-focused release: a 10k-image dataset trained for 10 epochs on an L4 went from 55 minutes to 31 minutes.

Faster, with no code change:

• −23% per training step at the default batch size, from the optimizer parameter-group merge.
• Up to −46% on predict() for large input frames, from moving uint8 widening onto the accelerator.
• 1.25–2.26× DataLoader throughput on input-bound runs, from packed targets and draft JPEG decoding.

New: a GPU batched linear-assignment solver for the matcher on CUDA, active automatically on eligible devices.

Six changes alter behavior for callers who change nothing. Read the Migration guide before you retrain:

• Validation evaluates one model per epoch instead of two, so val/* metrics now describe the EMA model.
• grad_accum_steps defaults to 1 instead of 4, moving the default effective batch from 16 to 4.
• Optimizer parameter groups collapse from one-per-parameter to one-per-hyperparameter.
• Dataset builders validate their config instead of silently falling back to wrong defaults.
• log_per_class_metrics defaults to False, dropping per-class keys from a default run.
• compute_val_loss defaults to "auto", computing val/loss only when something consumes it.

This release doesn't restate content already shipped in the 1.9.1–1.9.4 patch releases (a separate maintenance line); see those sections if you're upgrading directly from 1.9.0.

✨ Spotlights

The seven changes most likely to affect you, either because they make code you already have faster, or because they change what that code does.

🏋️ Training steps are about 23% faster at the default batch size

No code change needed. get_param_dict used to build one optimizer parameter group per trainable tensor (465 groups on rfdetr-nano), which disabled AdamW's foreach/fused multi-tensor batching entirely. Grouping by the (lr, weight_decay) pair each parameter already carried collapses that to 28 groups, and every parameter keeps exactly the learning rate and weight decay it had before.

At the default batch_size=4, a full RFDETR.train() step on an L4 falls from 223.17 ms to 170.77 ms on rfdetr-nano, a drop of −23.5%, with rfdetr-small matching at −23.3%. The saving is a fixed ~50–60 ms per optimizer step rather than a fraction of it, so its share shrinks as the batch grows: −17.7% at batch 8, −9.1% at batch 16. Weights are bit-identical across the two groupings. Every configuration is tabulated under Training. ([#1409])

End to end, with the release's evaluation and loader work stacked on top: a 10,000-image dataset trained for 10 epochs on an L4 went from 55 minutes on 1.9.4 to 31 minutes on 1.10.0, 44% less wall clock for the same run. Full COCO2017 trains at about 10 minutes per epoch on an RTX 6000.

🎯 predict() is up to 46% faster on large frames

Also free. predict() used to widen uint8 pixels to float32 before the host-to-device copy, so the bus carried four bytes per channel where one would do; it now transfers a zero-copy uint8 view and widens on the accelerator.

On a 2160×3840 PIL frame, RFDETRNano.predict() on an L4 falls from 77.022 ms to 41.319 ms, a drop of −46.4%, while a 640×640 frame gains only −6.5%. The gain scales with the input frame rather than the model, since the model does identical work either way; an RTX 4060 reaches −61% on the same 4K frame. Pixels arriving at the model stay byte-identical to torchvision's to_tensor. Every frame size and input type is tabulated under Inference, alongside five further predict() changes in the same release. ([#1415])

✅ Validation evaluates one model per epoch

Every validation epoch used to run two full forward passes, EMA weights and non-EMA, and report val/* from the non-EMA pass. It now runs one, so the reported metrics describe the model you actually ship and validation costs half the forward work.

from rfdetr import RFDETRSmall
from rfdetr.config import TrainConfig

# 1.10.0 default: one forward pass (EMA), removes the second non-EMA validation pass
model = RFDETRSmall()
model.train(dataset_dir="my_dataset", epochs=50)

# Restore the previous two-forward comparison
model.train(dataset_dir="my_dataset", epochs=50, train_config=TrainConfig(eval_base_model=True))

🔢 grad_accum_steps now defaults to 1, not 4

The old default quietly multiplied your batch: batch_size=4 really trained at an effective 16. Accumulation is now opt-in, so TrainConfig() means what it says; any run that relied on the old default needs the explicit argument back to keep its optimization schedule.

TrainConfig()  # effective batch = 4 (was 16)
TrainConfig(grad_accum_steps=4)  # reproduces the 1.9.x default (effective batch 16)

🧮 GPU batched linear-assignment solver

The Hungarian matcher solved each decoder layer's assignment on the CPU via SciPy, paying a device-to-host sync per layer. On eligible CUDA devices the new torch-hungarian backend solves the batched problem on the GPU instead, and SciPy still handles every other device and any problem above the size limit.

pip install 'rfdetr[train]'  # pulls torch-hungarian==0.1.0rc0
# No code change needed: the matcher picks the GPU solver automatically on
# CUDA + compute capability >= 8.0 + torch >= 2.4, falling back to SciPy elsewhere.

🧩 Optimizer parameter groups merge (465 → 28 groups for rfdetr-nano)

Same change as the training-step spotlight above, repeated here because it is also breaking: anything that indexes optimizer parameter groups positionally, or sizes a scheduler list to their count, now sees 28 groups where it saw 465.

# A custom lr_scheduler_kwargs list sized to the old per-parameter group count
# needs resizing to the new (much smaller) merged group count.
TrainConfig(lr_scheduler_kwargs={"lr_lambda": [fn] * 28})  # was * 465

🗂️ Dataset builders now require a complete pipeline-option namespace

These low-level builders used to fill a missing pipeline option from a default that did not necessarily match your model, so a wrong patch_size produced a quietly mistrained model instead of an error. Only direct callers are affected: .train() and RFDETRDataModule always passed a complete namespace.

# Before: missing fields silently fell back to (sometimes wrong) defaults
build_roboflow_from_coco(args=partial_namespace)

# After: raises unless every pipeline option is set
partial_namespace.square_resize_div_64 = True
partial_namespace.segmentation_head = False
partial_namespace.multi_scale = True
partial_namespace.expanded_scales = True
partial_namespace.do_random_resize_via_padding = False
partial_namespace.patch_size = model_config.patch_size
partial_namespace.num_windows = model_config.num_windows
build_roboflow_from_coco(args=partial_namespace)

🔄 Migration guide

Six changes alter behavior for callers who change nothing. What to do about each:

Change	What to do
grad_accum_steps defaults to 1 (was 4)	Pass grad_accum_steps=4 to keep the 1.9.x effective batch of 16
Validation evaluates one model per epoch	val/mAP_*, val/mAR, val/loss now describe the evaluated model (EMA by default); pass eval_base_model=True for the old two-forward comparison
Optimizer parameter groups merge by hyperparameter	Resize any custom lr_scheduler_kwargs list sized per-parameter (465 groups for rfdetr-nano) to the merged count of 28
Dataset builders require a complete pipeline-option namespace	Direct callers must set every pipeline option; read patch_size/num_windows from your ModelConfig rather than hardcoding them
log_per_class_metrics defaults False (was True)	Pass log_per_class_metrics=True if a dashboard consumes per-class keys
compute_val_loss defaults "auto" (was True)	"auto" keeps val/loss whenever something consumes it; pass True to force it

TrainConfig.eval_ema_only is deprecated (removal in v1.13) and superseded by eval_base_model. Checkpoints written with the old per-parameter optimizer layout are regrouped automatically on load, so resuming a 1.9.x run needs no action.

See MIGRATION.md for the full "Upgrade 1.9 → 1.10" guide with before/after code for each of these.

⚡ Performance

Twenty-seven pull requests in this release are performance work, and none of them trade accuracy for speed. Every one landed with an output-parity check: bit-identical tensors, byte-identical public detections, or an unchanged COCO metric. The ones with a stage-level or end-to-end number are detailed below; the remaining component-level wins are listed with their measured effect in the Extensive changelog.

👀 At a glance

Every row is measured rather than projected: the first two come from full training runs, the rest from the benchmark in the PR nam...

v1.9.4: Export & Aug Correctness Fixes

RF-DETR 1.9.4 is a maintenance release: no public API removed or renamed, no new public API — the new arguments below sit on private ONNX/TFLite reference helpers. It fixes seven bugs — two in export (ONNX/TFLite background-logit-slot assumption, a keypoint tensor misidentified as a segmentation mask), two silent annotation-corruption bugs in augmentation (keypoint flip with empty flip-pairs, Albumentations TimeReverse/SquareSymmetry), one training-correctness bug (BestModelCallback scoring PyTorch Lightning's sanity-check pass as a real epoch), one non-square training resize bug, and one environment-dependent TFLite export failure.

Users exporting sparse-ID COCO checkpoints — including the official pretrained weights — or legacy background-first keypoint checkpoints, training keypoint models with the pydantic-default empty keypoint_flip_pairs, using custom Albumentations TimeReverse/SquareSymmetry configs, starting a new training run seeded from pretrain_weights, or training non-square models benefit most from this release. See the migration guide below — three behavior changes may need action on upgrade.

✨ Spotlights / highlights

Export: configurable background logit slot (#1397)

The ONNX/TFLite reference inference helpers assumed the background class always sits at the final logit index. Sparse-ID COCO checkpoints and legacy background-first keypoint checkpoints decoded wrong. A new background_class_id argument on the private _run_inference helpers makes the assumption explicit — -1 (default) preserves current behavior, None keeps every logit slot, 0 supports legacy background-first keypoint checkpoints. The -1 default is kept for backward compatibility; it still mis-decodes the checkpoints named above, including the official pretrained COCO weights (a real foreground category occupies the final slot there) — pass background_class_id=None explicitly for those. The same commit also replaces the TFLite helper's old guess-any-lone-rank-4-output-is-a-mask behavior with a new rank4_output argument, defaulting to None — a keypoint export's pred_keypoints tensor could previously get silently upsampled into Detections.mask.

Keypoint annotations no longer silently corrupted on flip (#1358)

RandomHorizontalFlip on the torchvision-native backend mirrored keypoint x-coordinates on every drawn flip, but only relabeled left/right joints if self.keypoint_flip_pairs:. With an empty list — the pydantic default — training samples got keypoints mirrored in position while keeping their original left/right label, with no warning. The flip is now dropped entirely for an empty-but-not-None keypoint_flip_pairs, matching the Albumentations backend's existing safety contract.

Albumentations TimeReverse/SquareSymmetry box and keypoint handling (#1386)

Custom Albumentations configs using TimeReverse had it treated as pixel-only — it flipped images while leaving boxes and keypoints unchanged. It now shares the same geometric-transform and replay-based keypoint handling as HorizontalFlip, and the no-pairs safety filter now also covers SquareSymmetry.

BestModelCallback no longer scores the PTL sanity check as a real epoch (#1357)

A positive validation score from PyTorch Lightning's pre-training sanity check — common when starting a new run initialized with pretrain_weights from a checkpoint pretrained on a different dataset — could get written out as the permanent "best" EMA checkpoint before a single real epoch ran, after which real training could never surpass it. EMA tracking now honors the same trainer.sanity_checking guard the regular checkpoint path already had. (This is distinct from PTL's own resume/ckpt_path restart, which PTL itself skips the sanity check for — resumed runs were never affected.)

🔄 Migration guide

No public API was removed or renamed. Three behavior changes may need action on upgrade:

• TFLite segmentation inference. The _run_inference reference helper no longer treats an anonymous rank-4 output as a mask. Pass rank4_output="masks" for a name-stripped segmentation export.
• Keypoint training with keypoint_flip_pairs=[] (the pydantic default). The default horizontal flip is now disabled instead of applied without relabeling. Provide left/right pairs to keep the augmentation.
• Non-square training. The crop branch no longer resamples through a fixed 384x384 intermediate, so the augmented pixel distribution differs from 1.9.3. Square training (the default for every shipped model config) is unchanged.

📝 Notable changes

🔧 Fixed

• ONNX and TFLite reference inference helpers now accept an explicit background_class_id; a new rank4_output argument replaces the old guess-any-lone-rank-4-output-is-a-mask behavior and defaults to None. (#1397)
• Non-square training resize no longer double-resamples crop-branch outputs through a fixed 384x384 intermediate. (#1383)
• Custom Albumentations TimeReverse no longer leaves boxes and keypoints unflipped while the image flips; the no-pairs keypoint safety filter now also covers SquareSymmetry. (#1386)
• TFLite export no longer fails when onnx2tf can't resolve onnxsim from a non-activated virtualenv; RF-DETR now temporarily adds the running interpreter's script directory to PATH during conversion. (#1366, fixes #1365)
• Torchvision-native training no longer silently mirrors keypoint positions without relabeling left/right joints when keypoint_flip_pairs=[]. (#1358)
• BestModelCallback no longer treats PyTorch Lightning's sanity-check pass as a real epoch's result. (#1357, fixes #1348)

🏆 Contributors

• Jesús Royeth (@JESUSROYETH) — export background-logit-slot fix, keypoint flip-pair safety fix, Albumentations TimeReverse/SquareSymmetry fix, BestModelCallback sanity-check fix.
• Aman Harsh (@amanharshx, LinkedIn) — TFLite export onnxsim-on-PATH fix.
• jirka (@Borda, LinkedIn) — non-square training resize double-resample fix.

---

Full changelog: 1.9.3...1.9.4

v1.9.3: Faster Predict, Correct YOLO Test-Split & Export Fixes

RF-DETR 1.9.3 is a maintenance release: no breaking changes, no new top-level public API. It fixes ten user-facing bugs and improves five behavior refinements, spanning training (YOLO test-split evaluation, auto-batch memory estimation, resumed-run metrics history, EMA epoch-boundary double-counting), inference (predict() no longer blocks per-image on a CUDA sync for CUDA-tensor inputs), export (ONNX/TFLite no longer silently drop multi-label detections, ONNX Runtime benchmark now honors --device), and segmentation (skip_blocks path now applies the learned feature projection it was missing during training).

Users training on YOLO-format datasets with a test split, exporting to ONNX/TFLite, or relying on batch_size="auto" benefit most from this release. Callers passing CUDA tensors directly to predict() get a small inference speedup from the sync fix; everyone should note the PostProcess tie-breaking change described below — non-breaking, but worth a glance if code depends on exact output order.

✨ Spotlights / highlights

predict() no longer blocks on a per-image CUDA sync for CUDA-tensor inputs (#1341)

RFDETR.predict()'s [0, 1] pixel-range validation used to sync the host after every single image. For callers passing CUDA tensors directly (the host-round-trip-free path), range-check results are now collected unsynced across the whole batch and resolved once, so later images' GPU work can overlap the sync from earlier ones. File-path and PIL inputs go through a CPU tensor conversion first and were never blocked by this sync, so they see no change here.

from rfdetr import RFDETRSmall

model = RFDETRSmall()
cuda_images = [img.cuda() for img in preprocessed_tensors]
# same call, same output — just no longer stalls the host once per CUDA-tensor image
detections = model.predict(cuda_images)

YOLO-format evaluate(split="test") now evaluates the real test split (#1329, #1343)

Previously, requesting the test split on a YOLO-format dataset silently evaluated valid/ instead. Now it resolves the real test/ split when present, falls back to valid/ with a logged warning when the split is genuinely absent, and raises when a test path is declared in data.yaml but unresolvable or invalid.

model.train(dataset_dir="my_yolo_dataset", ...)
model.evaluate(split="test")  # now evaluates test/, not valid/

PostProcess selection is now deterministically tie-broken (#1320)

Detection selection switched from torch.topk to a stable argsort, so ties resolve the same way every run: descending score, then ascending flattened query/class index — the same rule now shared with the torch-free export decoders (both sides changed together in this PR). See the migration guide below if your code depends on tie order.

ONNX/TFLite export no longer silently drops multi-label detections (#1320)

The ONNX and TFLite reference decoders used a per-query argmax, keeping at most one class per query — any query scoring above threshold on more than one class quietly lost the rest. Both decoders now flatten (queries, classes) pairs and take the top-scoring ones before thresholding, mirroring PostProcess's own selection rule.

batch_size="auto" now accounts for AdamW's optimizer-state memory (#1342)

The auto-batch probe previously ignored AdamW's exp_avg/exp_avg_sq state, so the probed batch size could overshoot real training memory and OOM on the first optimizer step. It now models that state via a shadow optimizer and warns when a non-AdamW optimizer is configured (the estimate may then be too conservative or too permissive, since it no longer directly applies).

🔄 Migration guide

No breaking changes in this release.

Three behavior changes are worth a second look — none require a code change to keep working:

• PostProcess detection ordering may shift on score ties. Selection now uses a stable argsort instead of torch.topk, so ties resolve by ascending flattened query/class index instead of topk's implementation-defined order. Same detections, same scores — only the order among exact ties can differ, so detections[0] may point at a different (but equally-scored) box than before. Ordering among tied scores was never a documented contract. PostProcess(num_select=<negative>) now raises ValueError at construction instead of being silently accepted — if you were relying on that being a no-op, pass a non-negative value instead. (This selection now sorts the full flattened query/class score axis instead of doing a partial top-k; unmeasured this release, but callers with very large query/class counts on the hot inference path may want to benchmark their own workload.)
• RFDETR.predict() shape-validation errors changed type. Passing a malformed-rank input together with include_source_image=True now raises a public ValueError with a shape message, where it previously surfaced an internal RuntimeError from permute(). If you were catching RuntimeError around this call, catch ValueError instead (or both, for compatibility across versions).
• RFDETREMACallback.on_train_epoch_end no longer exists as an override point. If you subclass RFDETREMACallback and rely on its own on_train_epoch_end running (or override it yourself), note the epoch-boundary EMA update it performed is gone (that's the #1319 fix above) — calls to super().on_train_epoch_end(...) still resolve fine via PyTorch Lightning's base Callback, but as a no-op rather than an EMA update.

📝 Notable changes

🌱 Changed

• HungarianMatcher's compact-path safety gate now computes its target-side half (box/label finiteness checks) once per training step instead of once per matcher() call. SetCriterion.forward invokes matcher() separately for the final layer, each auxiliary decoder layer, and the encoder layer with the same targets, so the target-side precheck is now precomputed once and reused across all of them, keyed on targets object identity plus pred_boxes dtype/device and num_classes (a mismatch triggers a fresh computation). Matching results are unchanged. Callers must not mutate targets in place between precompute and reuse — the identity check cannot detect that. (#1340)
• Per-class confidence-threshold sweeps in evaluation are now O(N log N) instead of O(T·N): one stable ascending sort per class plus np.searchsorted into precomputed suffix sums replaces a full rescan per threshold. NaN scores are explicitly masked so they never count as "above threshold". Results are unchanged. (#1339)
• RFDETR.predict() no longer blocks the host on a per-image CUDA sync for its [0, 1] pixel-range validation. The range-check tensors are now collected unsynced across all images and resolved to Python booleans once, after every image's conversion, range check, and transfer have already been queued, so later images' GPU work can overlap the sync. Error-message precedence per image is unchanged. A malformed-rank input combined with include_source_image=True now raises a public ValueError with a shape message, where it previously surfaced an internal RuntimeError from permute(). (#1341)
• Transformer.forward's two-stage query selection now gathers the torch.topk-selected rows before running the bbox-delta MLP (enc_out_bbox_embed), not after — the MLP is pointwise with no cross-token mixing, so it only ever needs at most num_queries rows that survive selection, not every one of the sum(H*W) encoder positions. (#1334)
• PostProcess box/mask/keypoint selection is now deterministically tie-broken: torch.argsort(..., stable=True) plus a slice replaces torch.topk, so ties resolve by descending score then ascending flattened query/class index — the same rule now shared with the torch-free export decoders (both sides changed together in this PR). Output ordering may differ from 1.9.2 when scores tie (same detections, different order; detections[0] may change), but ordering among equal scores was never contractual. PostProcess(num_select=<negative>) now raises ValueError at construction instead of being silently accepted. (#1320)

🔧 Fixed

• YOLO-format evaluate(split="test") now evaluates the real test/ split, where it previously silently fell back to valid/. When no resolvable test split exists, it falls back to valid/ with a logged warning; when a test path is declared in data.yaml but unresolvable or invalid, it raises instead of silently relabeling as validation data. COCO-format datasets are unaffected — they never attempt a test fallback. (#1329, #1343)
• metrics.csv training history now survives a resumed run instead of being wiped on the first write after resume. (#1325)
• SegmentationHead's skip_blocks branch now applies the learned spatial_features_proj projection, matching the non-skip branch — the projection was being skipped entirely. Affects segmentation training loss trajectories only (encoder auxiliary mask supervision); the export path already applied this projection unconditionally before this change, and predict() outputs and exported models are unchanged. (#1331)
• Non-fin...

v1.9.2: Fixed Roboflow COCO & perf. optim

RF-DETR 1.9.2 is a fix-and-performance release: one breaking change, no new public APIs. Hierarchical COCO datasets — most commonly Roboflow exports carrying a synthetic unannotated grouping category — now get a correct, shared label mapping across train/valid/test, closing a case where per-class metrics could silently corrupt on the smaller split. The matcher's compact cost-matrix path is faster and lower-memory on real batches, training-resume from BestModelCallback's lightweight checkpoints now restores per-callback state instead of restarting cold, and a handful of training/eval fixes round it out.

✨ Spotlights / highlights

Breaking: hierarchical COCO datasets get a correct, shared label space

Roboflow COCO exports prepend a synthetic root category (unannotated, supercategory: "none") that every real class lists as its own supercategory. It never carried annotations but still consumed a label slot — training such a dataset built an N+1-class head instead of N.

from rfdetr.datasets.coco import filter_parent_categories, annotated_category_ids

categories = [
    {"id": 0, "name": "project-root", "supercategory": "none"},
    {"id": 1, "name": "car", "supercategory": "project-root"},
    {"id": 2, "name": "truck", "supercategory": "project-root"},
]
anns = {"annotations": [{"category_id": 1}, {"category_id": 2}]}

kept = filter_parent_categories(categories, annotated_category_ids(anns))
[c["name"] for c in kept]
# -> ['car', 'truck']  — the unannotated root no longer takes a label slot

For datasets where a parent category does carry its own annotations, label indices are now derived once from the train split and shared into valid/test, so a grouping category annotated in one split but not another no longer shifts that split's label indices independently of the others.

⚠️ Checkpoints trained before 1.9.2 keep their old head width and label ordering. Evaluating one against a re-filtered dataset misaligns per-class metrics (an existing UserWarning fires). See the migration notes below.

Matcher: faster and lower-memory on real COCO batches

HungarianMatcher's detection-only cost matrix is now built padded to each batch's max(T_i) target count instead of the cross-image sum(T_i), when a fast eligibility check passes — identical results on eligible batches, automatic full-computation fallback otherwise.

	before	after	change
Matcher time	—	—	~51% faster
Peak CUDA memory	—	—	~73–76% lower
Training step (A100)	288.364 ms	232.457 ms	—

The saving scales with target-count evenness across a batch — a batch where nearly all targets sit in one image sees little to no improvement.

Training resume restores per-callback state

Resuming from one of BestModelCallback's four lightweight checkpoints (checkpoint_best_regular.pth, checkpoint_best_ema.pth, checkpoint_best_total.pth, last_ema.pth) previously restarted best-score tracking, EMA, and early-stopping cold, silently. It now restores that state:

model.train(
    dataset_dir="my_dataset",
    resume="output/checkpoint_best_ema.pth",
    output_dir="output",  # must match the original run for best-score restore
)

📝 Notable changes

🌱 Changed

• HungarianMatcher compact-path speedup — detection-only cost matrix padded to max(T_i) instead of sum(T_i); ~51% faster matcher, ~73–76% lower peak CUDA memory on real COCO batches. Compact path also copies only diagonal cost blocks to CPU and batches its safety-gate sweeps into one sync. (#1297, #1281, #1312)
• Deterministic-algorithm coverage — seed_all() now also enables torch.use_deterministic_algorithms(True, warn_only=True); ops without a deterministic kernel warn at execution time instead of raising. (#1307)
• predict() — pins CPU image tensors before the CUDA transfer. (#1313)
• Two-stage query selection — avoids materialising repeated top-k gather indices. (#1278)
• Evaluation matching — counts labels on host instead of device. (#1276)
• Keypoint postprocessing — skips redundant CUDA presence checks. (#1282)

🔧 Fixed

• BestModelCallback resume — resuming from a lightweight checkpoint (checkpoint_best_*.pth, last_ema.pth) now restores per-callback state instead of restarting best-score tracking, EMA, and early-stopping cold; a warning distinguishes checkpoints that intentionally omit optimizer state from ones that predate callback-state persistence entirely. (#1318)
• Rich epoch progress bar — no longer corrupted or duplicated by training-time log calls under RichProgressBar(leave=True). (#1316)
• _kp_active_mask false-positive warning — loading a pre-keypoint-support checkpoint no longer warns about a deterministic schema buffer as a missing parameter. (#1302)
• Deferred CUDA-device move — an index-less torch.device("cuda") is normalised to the current device index before comparison, fixing spurious re-moves of every parameter on every call. (#1311)
• Legacy query-embedding fallback — only warns when it actually truncates weights. (#1301)
• eval_ema_only — no longer logs an empty validation pass when the base metric is empty; EMA metrics (val/ema_mAP_50_95, val/ema_mAP_50, val/ema_mAR, per-class AP, val (ema) summary table) are computed instead, and val/F1 is no longer silently dropped. (#1289)
• ModelContext.reinitialize_detection_head — raises a clear RuntimeError instead of AttributeError: 'NoneType' after RFDETR.inference(inplace=True) clears the weights. (#1283)
• evaluate() — builds its datamodule from the resolution-override config. (#1280)

⚠️ Breaking Changes

• Hierarchical COCO label filtering — unannotated grouping categories no longer consume a label slot; train/valid/test now share one label mapping derived from train. Checkpoints trained before this change need retraining against the new label space — see Migration guide. (#1303)

🏆 Contributors

• @JESUSROYETH — matcher perf (padded cost matrix, diagonal-only CPU copy, batched safety-gate sync), BestModelCallback resume-state restore, Rich progress-bar fix, eval_ema_only fix, and five other training/inference fixes
• @atikulmunna (LinkedIn) — guarded ModelContext.reinitialize_detection_head against cleared weights; strict-typed rfdetr.detr and datasets.yolo module boundaries
• @Jonathan-Jesni — strict-typed rfdetr.datasets.save_grids module boundary
• @Borda (LinkedIn) — hierarchical COCO label-filtering fix and release maintenance

---

Full changelog: 1.9.1...1.9.2

v1.9.1: Faster Inference & Export, Correct Export Resize

RF-DETR v1.9.1 is a maintenance release that makes inference and export both faster and more correct — with no breaking changes and no code changes required. Segmentation post-processing is 2.6–3.0× faster, predict() is ~20% faster at 1080p by skipping masks it would discard, and on-device ExecuTorch/XNNPACK is 2.5× faster. Exported ONNX/TFLite/INT8 models now resize exactly like predict(), closing a silent confidence and calibration drift. A handful of install fixes round it out on the Python-version edges.

✨ Spotlights / highlights

Segmentation post-processing 2.6–3.0× faster

PostProcess no longer materialises a repeated int64 gather index (21–84 MiB per image for the mask head) — it selects rows with index_select/expand instead. Output is bit-for-bit identical.

mask head, K=300	before	after	speedup
96²	8.79 ms	3.40 ms	2.6×
192²	79.99 ms	27.09 ms	3.0×

predict() ~20% faster at 1080p

Segmentation masks below the caller's threshold are dropped before upsampling instead of after, so ~97% of the resize work on typical COCO images disappears. The saving scales with image area (bigger at 4K, neutral at 640 px); output is unchanged.

model.predict(image, threshold=0.5)  # only surviving masks are upsampled

ExecuTorch / XNNPACK 2.5× faster

Export recombines the addmm operations XNNPACK leaves undelegated back into aten.linear. RFDETRNano on Apple silicon: 119.9 → 48.3 ms median; outputs match to ~1e-4.

model.export(format="executorch", backend="xnnpack")

Exported models resize exactly like predict()

ONNX inference, TFLite inference, INT8 calibration, and the benchmark path now use predict()'s resize convention (bilinear, half-pixel centers, antialias=False) rather than PIL's antialiased filters, which diverged on downscale.

⚠️ If you ship INT8 TFLite models, re-export them to recalibrate against the corrected pixel distribution.

📝 Notable changes

🌱 Changed

• Segmentation post-process — index_select/expand replace a repeated int64 gather index; 2.6–3.0× faster at head resolution, output unchanged. (#1268)
• predict() mask upsampling — masks below threshold are discarded before upsampling; ~20% faster at 1080p, output unchanged. (#1265)
• ExecuTorch export — AddmmToLinearTransform recombines undelegated addmm into aten.linear; ~2.5× faster XNNPACK inference. (#1262)

🔧 Fixed

• Export resize parity — ONNX/TFLite inference, INT8 calibration, and benchmark paths now match predict()'s resize convention; re-export INT8 TFLite models to recalibrate. (#1269)
• [onnx] / [executorch] install — extras are gated to interpreters that ship wheels, fixing [onnx] on Python 3.10 and [executorch] on Python 3.14. (#1267)
• Kornia range params — GaussianBlur/GaussNoise builders accept a scalar or a (min, max) pair, matching the Albumentations path. (#1255)
• keypoint_flip_pairs — detection-only datasets with a custom aug_config keep horizontal-flip augmentations by passing None instead of []. (#1248)
• uv sync — an executorch/tflite extra conflict that blocked dev-environment creation is resolved. (#1253)

📚 Documentation

• Updated ONNX, TFLite, ExecuTorch, CoreML, and INT8 calibration examples to use the same tensor-first, antialias=False preprocessing as predict(). The TFLite example now selects dets and labels outputs by name. (#1269)
• Corrected keypoint model Params (126.4 → 40.7 M), added a Params column to the keypoint benchmarks, and clarified that the SAM 3 RF100-VL result is author-reported, not measured by RF-DETR/SAB. (#1258, #1261)
• Fixed the ONNX Runtime example to include the missing decode step. (#1251)
• Added keypoint coverage to llms.txt / llms-full.txt. (#1260)

---

🏆 Contributors

• @JESUSROYETH — faster segmentation post-processing, threshold-aware mask upsampling, and export resize parity
• @chmjkb — 2.5× faster ExecuTorch / XNNPACK inference (React Native ExecuTorch team)
• @Vedanshu7 — gated the [onnx]/[executorch] extras to interpreters that ship wheels
• @adhavan18 (LinkedIn) — scalar-or-pair range parameters for the Kornia augmentation builders
• @unaxEtxeberriaBieleDigital — fixed uv sync dev-environment resolution
• @isaacrob (LinkedIn) — corrected keypoint benchmark Params and expanded llms.txt coverage
• @Borda (LinkedIn) — keypoint_flip_pairs fix, ONNX docs correction, and release maintenance
• @atikulmunna (LinkedIn) — stricter type-checking across variants and model_weights

---

Full changelog: 1.9.0...1.9.1

v1.9.0: Configurable Training, ExecuTorch & CoreML Export

RF-DETR v1.9.0 makes training more configurable and deployment more portable. The training optimizer and LR scheduler are now pluggable (any torch.optim name, dotted import path, or callable), keypoint/pose models can train under multi-GPU and multi-node DDP, and models export to ExecuTorch (.pte) and native CoreML (.mlpackage) for on-device inference, alongside an improved TensorRT export path. Checkpoint loading is also hardened: RFDETR.from_checkpoint() now defaults to safe (weights_only=True) deserialization. The default augmentation backend changes in this release — see the migration guide before upgrading if you rely on the previous default.

✨ Spotlights / highlights

ExecuTorch export

model = RFDETRSmall()
model.export(format="executorch", backend="xnnpack")  # or "coreml", "qnn"

Native CoreML export

model = RFDETRSmall()
model.export(format="coreml", coreml_precision="float16")  # .mlpackage, no ONNX intermediary

Multi-GPU / multi-node keypoint training

RFDETRKeypointPreview().train(
    dataset_dir="...",
    strategy="ddp",
    devices=4,
    grad_accum_steps=1,
)

Configurable optimizer & LR scheduler

from rfdetr.config import TrainConfig

TrainConfig(
    optimizer="pytorch_optimizer.Lion",
    lr_scheduler="torch.optim.lr_scheduler.OneCycleLR",
    lr_scheduler_kwargs={"max_lr": 1e-3, "total_steps": 10_000},
)

Safe checkpoint loading by default

model.from_checkpoint("checkpoint.pth")  # now weights_only=True by default
model.from_checkpoint("legacy_checkpoint.pth", trust_checkpoint=True)  # opt-in for trusted files

🔄 Migration guide

Breaking changes

albumentations and kornia extras merged into augment

Old extra	New extra
rfdetr[train] (implied albumentations)	rfdetr[train,augment]
rfdetr[kornia]	rfdetr[augment]

There is no [kornia] compatibility alias — pip install 'rfdetr[kornia]' fails after upgrading.

Default resize interpolation changed — pixel values and mAP may shift. The default resize backend changed from Albumentations to torchvision. To restore the previous behaviour:

pip install 'rfdetr[augment]'

from rfdetr.datasets.aug_configs import AUG_CONFIG
train_config = TrainConfig(aug_config=AUG_CONFIG, ...)

Installing rfdetr[augment] is not by itself sufficient to pin behaviour — pass augmentation_backend="torchvision" explicitly to pin it regardless of what's installed.

Unrecognised train() kwargs now raise ValueError — TrainConfig uses extra="forbid"; a typo'd kwarg that previously trained silently with defaults now raises immediately with a suggestion.

Removed

Deprecated in earlier releases, removed as of v1.9:

• rfdetr.util.* and rfdetr.deploy.* import paths (deprecated since v1.6) → rfdetr.utilities.*, rfdetr.assets.coco_classes, rfdetr.training.drop_schedule, rfdetr.training.param_groups, rfdetr.visualize.data, rfdetr.models.heads.segmentation, rfdetr.export
• build_namespace(model_config, train_config) (deprecated since v1.7) → build_model_from_config / build_criterion_from_config
• load_pretrain_weights(nn_model, model_config, train_config)'s train_config argument (deprecated since v1.7) → call (nn_model, model_config)
• start_epoch, do_benchmark, callbacks kwargs on .train()/.evaluate() (deprecated since v1.7) → resume=, rfdetr.export.benchmark, PTL Callback objects respectively
• Misplaced config fields — TrainConfig.{group_detr, ia_bce_loss, segmentation_head, num_select} → ModelConfig; ModelConfig.cls_loss_coef → TrainConfig (deprecated since v1.7)
• RFDETRLarge's silent fallback to RFDETRLargeDeprecatedConfig on checkpoint/config incompatibility — now raises the original error; use RFDETRLargeDeprecated directly for legacy Large weights

Deprecated in v1.9 → Remove in v1.11

• RFDETR.optimize_for_inference() renamed to RFDETR.inference() (same signature) — old name kept as an alias, emits FutureWarning
• TrainConfig.lr_drop and lr_min_factor — pass through lr_scheduler_kwargs instead

Full details, including code before/after examples for every item above: docs/getting-started/migration.md → "Upgrade 1.8 → 1.9".

📝 Notable changes

🚀 Added

• Native CoreML export — RFDETR.export(format="coreml") produces a .mlpackage (mlprogram, iOS 16+) directly from torch.export, no ONNX intermediary; coreml_precision="float32"|"float16" controls compute precision. Install with pip install 'rfdetr[coreml]' (macOS only, coremltools>=8.0,<10.0). Distinct from ExecuTorch's format="executorch", backend="coreml" .pte path. (#1235, #1244)
• ExecuTorch (.pte) export — RFDETR.export(format="executorch") for XNNPACK CPU, Core ML, and experimental Qualcomm QNN backends; static-shape deformable-attention export, fail-fast validation for unsupported dynamic batching. (#1142, #1231, #1237)
• TensorRT export improvements — RFDETR.export(format="tensorrt") (alias "trt") builds .trt engines in-process; configurable fp16: bool = True precision with automatic FP32 fallback. (#853, #1231)
• RFDETR.evaluate(*, split="test"|"val", **kwargs) — runs COCO evaluation (mAP, mAR, macro-F1, per-class AP) on the in-memory model without a checkpoint reload. (#1134)
• Multi-GPU / multi-node keypoint (pose) training under DDP — keypoint models train with strategy="ddp"/"auto", devices>1, num_nodes>1. Sharded strategies (FSDP/DeepSpeed) remain unsupported and now fail with a clear error. (#1232)
• Configurable training optimizer — TrainConfig.optimizer: str | Callable = "adamw" plus optimizer_kwargs. (#1006)
• Configurable LR schedulers — TrainConfig.lr_scheduler: str | Callable = "step" plus lr_scheduler_kwargs, lr_scheduler_interval, lr_scheduler_monitor; end-to-end ReduceLROnPlateau support. (#1217)
• TrainConfig.scale_jitter: bool = True — independent control of the resize→crop→resize training branch, decoupled from aug_config. (#1037)
• Torchvision-native augmentation backend + selectable backends — AugmentationBackend.TV/.ALBU/.KORNIA; augmentation_backend="torchvision" always pins torchvision. (#1112)
• Mask-aware dataset grids — instance-segmentation label validation renders masks in save_grids. (#1014)
• TrainConfig.eval_ema_only: bool = False — opt-in EMA-only evaluation. (#1225)

⚠️ Breaking Changes

• Augmentation backend default — training/validation/export now resolve to torchvision-native transforms unless Albumentations is installed; [train] no longer bundles Albumentations/Kornia. See migration guide. (#1112)
• Unrecognised train()/evaluate() kwargs now raise ValueError instead of silently training with defaults. (#1178)

🌱 Changed

• Always save EMA-named checkpoints (checkpoint_best_ema.pth, last_ema.pth) when monitor_ema is set; checkpoint_best_total.pth records provenance. (#1216)
• Epoch-level train loss shown in the training progress bar. (#1211)
• Improved training performance — foreach EMA update, gated validation loop, TF32 enabled. (#1226)
• Improved evaluation performance — removed per-iteration GPU sync in matching. (#1225)
• Matched-pair IoU targets in the loss computation now use new O(N) elementwise_box_iou/elementwise_generalized_box_iou helpers instead of building the full NxN pairwise matrix and reading its diagonal, reducing peak GPU memory during loss calculation. (#1245)
• [tensorrt] no longer installs pycuda — it's only needed for TRTInference's async benchmarking mode, now under the separate [tensorrt-bench] extra. The standard export→engine path is unaffected. (#1246)

🗑️ Deprecated

• RFDETR.optimize_for_inference() → RFDETR.inference() (same signature). (#1212)
• TrainConfig.lr_drop / lr_min_factor → lr_scheduler_kwargs. (#1217)

❌ Removed

• rfdetr.util.*, rfdetr.deploy, build_namespace(), load_pretrain_weights()'s train_config arg, start_epoch/do_benchmark/callbacks train() kwargs, misplaced TrainConfig/ModelConfig fields, RFDETRLarge's silent legacy-config fallback — all deprecated since v1.6/v1.7, removed as scheduled. (#1218)
• [kornia] pip extra — folded into [augment], no compatibility alias. (#1142, #1112)

🔧 Fixed

• First predict()/inference() call no longer silently breaks subsequent backward() gradients. (#1178)
• Optimized keypoint models correctly return sv.KeyPoints. (#1210)
• predict() resize antialias disabled to match training/val preprocessing. (#1206)
• ONNX export at non-native resolution no longer crashes. (#1207)
• Keypoint DDP training no longer hangs on the out-of-schema loss guard (graph-connected zeros instead of detached). (#1232)
• window_block_indexes override now correctly forwarded to DINOv2. (#1224)
• Non-square Albumentations training resize no longer inflates every image to max_size. (#1112, #1183)
• RFDETR.from_checkpoint(..., trust_checkpoint=True) now actually works — it previously bypassed the safe-load check only for the checkpoint's metadata read; model construction silently reloaded the same file without the caller's trust setting and raised anyway. (#1239)
• Segmentation evaluation now resizes ground-truth masks directly to each prediction's own pixel grid instead of each image's original resolution, removing a lossy round trip vs. the mask head's native grid — segm mAP is computed on consistent pixel grids. (#1241)
• pip install 'rfdetr[onnx]' (and [tflite]) no longer hangs building onnxsim from source on CPython 3.11/3.13 and...

v1.8.3: Correct keypoint flip augm.

📋 Summary

RF-DETR 1.8.3 is a focused patch release with one new capability and three correctness fixes. The headline addition is optimize_for_inference(inplace=True) — a memory-efficient inference path that skips the deep-copy of the base model, useful on memory-constrained GPUs and edge devices. On the bug-fix side: keypoint horizontal-flip augmentation now correctly swaps left/right pairs (previously labels were corrupted on flip); bounding boxes are clamped to image bounds so objects at image edges no longer produce negative or out-of-frame coordinates; and default loss coefficients for segmentation and keypoint fine-tuning are restored to their intended values after a regression introduced in v1.7.

✨ Spotlights

In-place inference optimization

optimize_for_inference(inplace=True) exports the model using the loaded weights directly — no deep-copy — reducing peak memory during the optimization step by roughly 0.5× model weight. After inplace optimization, the base model is cleared: export() raises RuntimeError and remove_optimized_model() issues a UserWarning and returns cleanly. The new is_optimized_inplace property lets you check the current state.

model = RFDETRSmall()

# Default: deep-copy kept, fully reversible
model.optimize_for_inference(compile=False, dtype="float16")
model.remove_optimized_model()  # works fine

# New: inplace, lowest possible memory — irreversible
model.optimize_for_inference(compile=False, inplace=True, dtype="float16")
print(model.is_optimized_inplace)  # True

Correct keypoint flip augmentation

Keypoint training now correctly swaps symmetric pairs during horizontal flip augmentation. CocoKeypointSchema and YoloKeypointSchema both gain a keypoint_flip_pairs field, populated automatically from keypoint names (COCO left/right convention) or from flip_idx (YOLO pose). infer_coco_keypoint_schema and infer_yolo_keypoint_schema are also re-exported from the public rfdetr.datasets namespace.

from rfdetr.datasets import infer_coco_keypoint_schema  # now public

schema = infer_coco_keypoint_schema("path/to/_annotations.coco.json")
print(schema.keypoint_flip_pairs)  # [1, 2, 3, 4, ...] — auto-inferred

model.train(dataset_dir=DATASET_DIR, epochs=50, keypoint_schema=schema)

Native COCO format (dataset_file="coco") is supported alongside "roboflow" and "yolo".

Bounding box coordinates clamped to image bounds

Predicted boxes are now guaranteed to lie within [0, width] × [0, height]. Model regression output is unbounded — objects near image edges could previously produce negative x1/y1 or x2/y2 values exceeding image dimensions. scale_fct is also cast to boxes.dtype to avoid dtype mismatch with fp16 inference.

Loss coefficient defaults corrected

Two training config defaults were silently wrong since v1.7:

• SegmentationTrainConfig.cls_loss_coef was 5.0 — now 1.0, restoring the effective weight from before the v1.7 TrainConfig ownership migration.
• KeypointTrainConfig.keypoint_nll_loss_coef was 0.5 — now 1.0, aligning the NLL term with all other keypoint loss weights.

If your current fine-tuned runs relied on the old defaults, pass the values explicitly to maintain continuity.

🔄 Migration guide

No breaking API changes in this release.

Segmentation fine-tuning — cls_loss_coef default change: if you fine-tune segmentation models without setting cls_loss_coef explicitly, your loss balance shifts. To reproduce prior runs:

from rfdetr.config import SegmentationTrainConfig

config = SegmentationTrainConfig(cls_loss_coef=5.0)

Keypoint fine-tuning — keypoint_nll_loss_coef default change: to reproduce prior runs:

from rfdetr.config import KeypointTrainConfig

config = KeypointTrainConfig(keypoint_nll_loss_coef=0.5)

Keypoint horizontal-flip augmentation now active: flip augmentation was disabled for keypoints in v1.8.1 because pair swapping was not yet implemented. With this release it is enabled automatically when keypoint_flip_pairs is non-empty. Datasets without left/right keypoint naming conventions infer zero pairs and skip ReplayCompose automatically — no action required.

📝 Notable changes

🚀 Added

• optimize_for_inference(inplace=True) — keyword-only arg on RFDETR.optimize_for_inference(); skips deep-copy for memory-constrained inference-only deployments. Requires compile=False. (#1089)
• RFDETR.is_optimized_inplace — property returning True after a successful inplace optimization. (#1089)
• CocoKeypointSchema.keypoint_flip_pairs and YoloKeypointSchema.keypoint_flip_pairs — flat list of horizontal-flip swap pairs, inferred from keypoint names (COCO) or flip_idx (YOLO). (#1164)
• infer_coco_keypoint_schema and infer_yolo_keypoint_schema re-exported from rfdetr.datasets (public namespace). (#1164)

🌱 Changed

• Horizontal flip detection in AlbumentationsWrapper now uses Albumentations ReplayCompose replay metadata instead of heuristic bbox-center mirroring; eliminates false positives. Falls back to alb.Compose with a UserWarning on albumentations <1.3. (#1164)
• Keypoint schema inference now supports native COCO format (dataset_file="coco") alongside "roboflow" and "yolo". (#1164)
• _keypoint_schema_cache key changed from dataset_dir to (dataset_file, dataset_dir) tuple to prevent cross-format cache collisions. (#1164)
• SegmentationTrainConfig.cls_loss_coef default: 5.0 → 1.0. (#1165)
• KeypointTrainConfig.keypoint_nll_loss_coef default: 0.5 → 1.0. (#1165)

🔧 Fixed

• Predicted bounding boxes clamped to [0, width] × [0, height] in PostProcess._postprocess_boxes(); scale_fct also cast to boxes.dtype for fp16 safety. (#1168)
• SegmentationTrainConfig.cls_loss_coef default corrected from 5.0 to 1.0 — restores the pre-v1.7 effective classification loss weight. (#1165)
• KeypointTrainConfig.keypoint_nll_loss_coef default restored to 1.0. (#1165)
• optimize_for_inference() flag ordering fixed: _optimized_inplace set before model.model = None so exception recovery sees correct state. (#1089)
• remove_optimized_model() now issues UserWarning and returns cleanly after inplace optimization instead of raising RuntimeError. (#1089)
• export() now raises RuntimeError immediately if called after inplace optimization. (#1089)

---

🏆 Contributors

• Jonas Pirner (@pirnerjonas) — added in-place inference optimization for memory-efficient deployment
• Alessandro Brunello (@Armaggheddon, LinkedIn) — fixed postprocessor to clamp predicted boxes to image bounds
• Jirka Borovec (@Borda, LinkedIn) — restored loss coefficient defaults, fixed keypoint flip-pair augmentation, restructured test layout

---

Full changelog: 1.8.2...1.8.3

v1.8.2: YOLO Pose Support, Active-First Keypoints

📋 Summary

RF-DETR 1.8.2 rounds out the keypoint detection feature set with YOLO pose dataset support (load Ultralytics YOLO pose datasets directly for training, no conversion needed), an active-first keypoint schema default that makes class IDs zero-based by default, and a new amp_dtype field for explicit fp16/bf16 mixed-precision control. Two new cookbooks land: instance segmentation fine-tuning and an inference latency benchmark. On the reliability side, a long-standing bug in from_checkpoint() that silently inflated num_classes by one is fixed, TensorRT ONNX export is unblocked for all model variants, and several inference correctness issues are resolved. Keypoint users: the default schema changed from background-first [0, 17] to active-first [17]. Checkpoint weights load unchanged; class IDs in inference output shift — see the migration guide below.

✨ Spotlights

YOLO pose keypoint datasets

Train keypoint models directly from Ultralytics YOLO pose datasets. Point dataset_dir at any dataset folder with a data.yaml containing kpt_shape — schema, keypoint names, and OKS sigmas are inferred automatically.

from rfdetr import RFDETRKeypointPreview
from rfdetr.config import KeypointTrainConfig

model = RFDETRKeypointPreview()
model.train(
    KeypointTrainConfig(
        dataset_dir="path/to/yolo-pose-dataset",
        epochs=50,
    )
)

Active-first keypoint schema — cleaner class IDs

Person is now at class_id=0 instead of class_id=1. Legacy checkpoints load without any changes — RF-DETR auto-detects the schema at load time. New schema utilities make conversion explicit when needed:

from rfdetr.utilities.keypoints import _is_bg_first_schema, _to_active_first

if _is_bg_first_schema(schema):
    schema = _to_active_first(schema)  # [0, 17] → [17]

amp_dtype on TrainConfig — pin fp16 or bf16

Stop relying on device auto-detection. Set the AMP dtype explicitly:

from rfdetr.config import TrainConfig

config = TrainConfig(dataset_dir="...", amp_dtype="fp16")  # force fp16
config = TrainConfig(dataset_dir="...", amp_dtype="bf16")  # force bf16
config = TrainConfig(dataset_dir="...", amp_dtype="auto")  # default, device heuristic

TensorRT ONNX export — now works

spatial_shapes in Transformer.forward() is now built from symbolic Shape ops, removing the ScatterND node that TensorRT rejected with "IScatterLayer cannot be used to compute a shape tensor". All RF-DETR variants can now export to TensorRT engines:

# After export(format="onnx"):
trtexec --onnx=model.onnx --saveEngine=model.engine

New cookbooks

Two new end-to-end notebooks:

• Instance segmentation fine-tuning (docs/cookbooks/fine-tune_segmentation.ipynb) — RFDETRSegSmall across seven diverse segmentation datasets with training metrics and sample previews.
• Inference latency benchmark (docs/cookbooks/inference-latency-benchmark.ipynb) — reproducible CPU/GPU throughput measurements across model sizes.

🔄 Migration guide

🌱 Changed: keypoint class IDs shift to zero-based — checkpoint weights unaffected

Affects RFDETRKeypointPreview / RFDETRKeypointPreviewConfig users.

Checkpoint weights load unchanged — RF-DETR auto-detects the schema from the checkpoint and aligns it at load time. No re-training or weight migration needed.

What breaks: class IDs in inference output shift. Person moves from class_id=1 to class_id=0. Post-processing code that hardcodes class IDs must update:

# Before (background-first [0, 17]: person was at class_id=1)
class_name = "person" if detection.class_id == 1 else "other"

# After (active-first [17]: person is at class_id=0)
class_name = "person" if detection.class_id == 0 else "other"

Schema-agnostic alternative (works with either schema):

class_name = detection.data["class_name"]

To keep the legacy schema, pass num_keypoints_per_class at construction time:

config = RFDETRKeypointPreviewConfig(num_keypoints_per_class=[0, 17])

📝 Notable changes

🚀 Added

• YOLO pose keypoint dataset support — load Ultralytics YOLO pose datasets (.yaml with kpt_shape) directly for keypoint training. Schema inferred via infer_yolo_keypoint_schema. (#1156)
• amp_dtype on TrainConfig — pin mixed-precision dtype to "auto" / "bf16" / "fp16". Invalid values degrade to "auto" with a UserWarning. (#1143)
• Keypoint schema utilities — is_bg_first_schema, to_active_first, to_bg_first, schemas_semantically_equal in rfdetr.utilities.keypoints (re-exported from rfdetr.utilities). (#1160)
• Instance segmentation fine-tuning cookbook (docs/cookbooks/fine-tune_segmentation.ipynb). (#1159)
• Inference latency benchmark cookbook (docs/cookbooks/inference-latency-benchmark.ipynb). (#1152)

🌱 Changed

• Default num_keypoints_per_class changed from [0, 17] to [17] in RFDETRKeypointPreviewConfig. Checkpoint weights load unchanged — RF-DETR auto-aligns the schema at load time. Class IDs in inference output shift (person moves from class_id=1 to class_id=0); post-processing code that hardcodes class IDs must update. (#1160)

🔧 Fixed

• from_checkpoint() now reads the correct num_classes — was class_embed.weight.shape[0] (including background), now shape[0] - 1. Prevented shape mismatches and silently added an extra output class to every fine-tuned checkpoint load. BestModelCallback._serialize_model_config also fixed. (#1158)
• TensorRT ONNX export unblocked — spatial_shapes built from symbolic Shape ops, removing the ScatterND that blocked TensorRT compilation. (#1155)
• HungarianMatcher now respects configured focal_alpha — was hardcoded to 0.25, misaligning bipartite matching cost with the actual focal loss. (#1147)
• Keypoint inference class_name corrected — predictions now carry the right class name for keypoint models. (#1151)
• predict() re-asserts eval mode — prevents silent train-mode inference for unoptimized models after the first call. (#1146)
• TFLite inference — preprocessing and mask decoder now match PyTorch predict(). (#1131)
• Python version mismatch in dependency overrides resolved. (#1137)

---

🏆 Contributors

Thanks to everyone who contributed to this release:

• Jirka Borovec (@Borda, LinkedIn) — active-first keypoint schema, YOLO pose support, checkpoint restore fix, segmentation + latency cookbooks
• Anatoly Ryabchenko (@ryabchenko-a) — amp_dtype field for explicit AMP dtype control
• Ruben (@RubenHaisma) — HungarianMatcher focal_alpha fix and predict() eval-mode re-assertion
• Isaac Robinson (@isaacrob, LinkedIn) — TensorRT-safe ONNX export via symbolic Shape ops
• Stefan Schneider (@hinogi) — dataset type refactoring and Python dependency fix
• Omkar Kabde (@omkar-334, LinkedIn) — TFLite inference preprocessing and mask decoder fix

---

Full changelog: 1.8.1...v1.8.2

v1.8.1: Stability Fixes

📋 Summary

RF-DETR 1.8.1 is a patch release with seven bug fixes and three quality-of-life improvements. The highlights: two compatibility crashes are fixed — one hitting environments that have tensorflow alongside NumPy 2.0, another hitting Albumentations 2.x users — and per-epoch metric tables show up again during training instead of being hidden behind the progress bar. No breaking changes, and nothing to migrate from v1.8.0.

🗒️ Notable changes

🌱 Changed

• pathlib.Path accepted for path parameters — path fields on TrainConfig and related config classes now accept pathlib.Path objects directly. (#1124)
• Horizontal flip disabled for keypoint training — flip augmentation is turned off for keypoint training for now, since it could produce incorrect labels without keypoint flip-pair reordering. (#1122)
• Training metric plots — improved plotting with optional error bands, AP@0.75 grouping, and configurable AP metric groups. (#1122)

🔧 Fixed

• TensorBoard + NumPy 2.0 crash — training falls back to CSV logging with a warning instead of crashing when TensorBoard can't be imported. (#1123)
• Albumentations 2.x compatibility — fixed a crash in horizontal-flip detection when using Albumentations 2.x. (#1126)
• Metric tables hidden by the progress bar — per-epoch metric tables now render correctly alongside the live training progress bar. (#1128)
• Keypoint checkpoint selection — best-checkpoint selection for keypoint models is now smoothed, avoiding spurious switches on noisy OKS metrics; smoothing also survives a training resume. (#1122)
• Keypoint encoder evaluation — fixed query routing in eval mode so keypoint predictions are computed correctly. (#1135)
• Group DETR train-time evaluation — fixed a crash during train-time evaluation in Group DETR mode. (#1122)
• DINOv2 backbone config compatibility — internal call sites updated from deprecated config.use_return_dict to config.return_dict. (#1135)

---

🏆 Contributors

Thanks to everyone who contributed to this release:

• Jirka Borovec (@Borda, LinkedIn) — keypoint encoder fix, metric table display, training stability, TensorBoard compatibility
• Juan Pablo Oberhauser (@jpoberhauser, LinkedIn) — Albumentations 2.x compatibility
• Brian Cong (@congbrian) — pathlib.Path config support
• Stefan Schneider (@hinogi) — type hint modernisation

---

Full changelog: 1.8.0...v1.8.1