v1.9.3: Faster Predict, Correct YOLO Test-Split & Export Fixes

RF-DETR 1.9.3 is a maintenance release: no breaking changes, no new top-level public API. It fixes ten user-facing bugs and improves five behavior refinements, spanning training (YOLO test-split evaluation, auto-batch memory estimation, resumed-run metrics history, EMA epoch-boundary double-counting), inference (predict() no longer blocks per-image on a CUDA sync for CUDA-tensor inputs), export (ONNX/TFLite no longer silently drop multi-label detections, ONNX Runtime benchmark now honors --device), and segmentation (skip_blocks path now applies the learned feature projection it was missing during training).

Users training on YOLO-format datasets with a test split, exporting to ONNX/TFLite, or relying on batch_size="auto" benefit most from this release. Callers passing CUDA tensors directly to predict() get a small inference speedup from the sync fix; everyone should note the PostProcess tie-breaking change described below — non-breaking, but worth a glance if code depends on exact output order.

✨ Spotlights / highlights

predict() no longer blocks on a per-image CUDA sync for CUDA-tensor inputs (#1341)

RFDETR.predict()'s [0, 1] pixel-range validation used to sync the host after every single image. For callers passing CUDA tensors directly (the host-round-trip-free path), range-check results are now collected unsynced across the whole batch and resolved once, so later images' GPU work can overlap the sync from earlier ones. File-path and PIL inputs go through a CPU tensor conversion first and were never blocked by this sync, so they see no change here.

from rfdetr import RFDETRSmall

model = RFDETRSmall()
cuda_images = [img.cuda() for img in preprocessed_tensors]
# same call, same output — just no longer stalls the host once per CUDA-tensor image
detections = model.predict(cuda_images)

YOLO-format evaluate(split="test") now evaluates the real test split (#1329, #1343)

Previously, requesting the test split on a YOLO-format dataset silently evaluated valid/ instead. Now it resolves the real test/ split when present, falls back to valid/ with a logged warning when the split is genuinely absent, and raises when a test path is declared in data.yaml but unresolvable or invalid.

model.train(dataset_dir="my_yolo_dataset", ...)
model.evaluate(split="test")  # now evaluates test/, not valid/

PostProcess selection is now deterministically tie-broken (#1320)

Detection selection switched from torch.topk to a stable argsort, so ties resolve the same way every run: descending score, then ascending flattened query/class index — the same rule now shared with the torch-free export decoders (both sides changed together in this PR). See the migration guide below if your code depends on tie order.

ONNX/TFLite export no longer silently drops multi-label detections (#1320)

The ONNX and TFLite reference decoders used a per-query argmax, keeping at most one class per query — any query scoring above threshold on more than one class quietly lost the rest. Both decoders now flatten (queries, classes) pairs and take the top-scoring ones before thresholding, mirroring PostProcess's own selection rule.

batch_size="auto" now accounts for AdamW's optimizer-state memory (#1342)

The auto-batch probe previously ignored AdamW's exp_avg/exp_avg_sq state, so the probed batch size could overshoot real training memory and OOM on the first optimizer step. It now models that state via a shadow optimizer and warns when a non-AdamW optimizer is configured (the estimate may then be too conservative or too permissive, since it no longer directly applies).

🔄 Migration guide

No breaking changes in this release.

Three behavior changes are worth a second look — none require a code change to keep working:

• PostProcess detection ordering may shift on score ties. Selection now uses a stable argsort instead of torch.topk, so ties resolve by ascending flattened query/class index instead of topk's implementation-defined order. Same detections, same scores — only the order among exact ties can differ, so detections[0] may point at a different (but equally-scored) box than before. Ordering among tied scores was never a documented contract. PostProcess(num_select=<negative>) now raises ValueError at construction instead of being silently accepted — if you were relying on that being a no-op, pass a non-negative value instead. (This selection now sorts the full flattened query/class score axis instead of doing a partial top-k; unmeasured this release, but callers with very large query/class counts on the hot inference path may want to benchmark their own workload.)
• RFDETR.predict() shape-validation errors changed type. Passing a malformed-rank input together with include_source_image=True now raises a public ValueError with a shape message, where it previously surfaced an internal RuntimeError from permute(). If you were catching RuntimeError around this call, catch ValueError instead (or both, for compatibility across versions).
• RFDETREMACallback.on_train_epoch_end no longer exists as an override point. If you subclass RFDETREMACallback and rely on its own on_train_epoch_end running (or override it yourself), note the epoch-boundary EMA update it performed is gone (that's the #1319 fix above) — calls to super().on_train_epoch_end(...) still resolve fine via PyTorch Lightning's base Callback, but as a no-op rather than an EMA update.

📝 Notable changes

🌱 Changed

• HungarianMatcher's compact-path safety gate now computes its target-side half (box/label finiteness checks) once per training step instead of once per matcher() call. SetCriterion.forward invokes matcher() separately for the final layer, each auxiliary decoder layer, and the encoder layer with the same targets, so the target-side precheck is now precomputed once and reused across all of them, keyed on targets object identity plus pred_boxes dtype/device and num_classes (a mismatch triggers a fresh computation). Matching results are unchanged. Callers must not mutate targets in place between precompute and reuse — the identity check cannot detect that. (#1340)
• Per-class confidence-threshold sweeps in evaluation are now O(N log N) instead of O(T·N): one stable ascending sort per class plus np.searchsorted into precomputed suffix sums replaces a full rescan per threshold. NaN scores are explicitly masked so they never count as "above threshold". Results are unchanged. (#1339)
• RFDETR.predict() no longer blocks the host on a per-image CUDA sync for its [0, 1] pixel-range validation. The range-check tensors are now collected unsynced across all images and resolved to Python booleans once, after every image's conversion, range check, and transfer have already been queued, so later images' GPU work can overlap the sync. Error-message precedence per image is unchanged. A malformed-rank input combined with include_source_image=True now raises a public ValueError with a shape message, where it previously surfaced an internal RuntimeError from permute(). (#1341)
• Transformer.forward's two-stage query selection now gathers the torch.topk-selected rows before running the bbox-delta MLP (enc_out_bbox_embed), not after — the MLP is pointwise with no cross-token mixing, so it only ever needs at most num_queries rows that survive selection, not every one of the sum(H*W) encoder positions. (#1334)
• PostProcess box/mask/keypoint selection is now deterministically tie-broken: torch.argsort(..., stable=True) plus a slice replaces torch.topk, so ties resolve by descending score then ascending flattened query/class index — the same rule now shared with the torch-free export decoders (both sides changed together in this PR). Output ordering may differ from 1.9.2 when scores tie (same detections, different order; detections[0] may change), but ordering among equal scores was never contractual. PostProcess(num_select=<negative>) now raises ValueError at construction instead of being silently accepted. (#1320)

🔧 Fixed

• YOLO-format evaluate(split="test") now evaluates the real test/ split, where it previously silently fell back to valid/. When no resolvable test split exists, it falls back to valid/ with a logged warning; when a test path is declared in data.yaml but unresolvable or invalid, it raises instead of silently relabeling as validation data. COCO-format datasets are unaffected — they never attempt a test fallback. (#1329, #1343)
• metrics.csv training history now survives a resumed run instead of being wiped on the first write after resume. (#1325)
• SegmentationHead's skip_blocks branch now applies the learned spatial_features_proj projection, matching the non-skip branch — the projection was being skipped entirely. Affects segmentation training loss trajectories only (encoder auxiliary mask supervision); the export path already applied this projection unconditionally before this change, and predict() outputs and exported models are unchanged. (#1331)
• Non-finite keypoint predictions no longer poison the shared box-head gradient via ref_wh, in both the decoder and encoder branches. The keypoint loss itself now also masks out non-finite predicted keypoints and non-finite target areas rather than letting them poison the loss. Not yet covered: the matcher's own keypoint cost still lacks the equivalent guard. (#1336)
• batch_size="auto" now models AdamW's optimizer-state memory in its probe, preventing an OOM on the first optimizer step when the probed batch size previously overshot real training memory. (#1342)
• ONNX Runtime benchmark now honors the requested --device, instead of always binding GPU 0. (#1346)
• Training metric plots now draw a legend on every subplot, not just the loss subplot. (#1335)
• ONNX/TFLite reference decode now mirrors PostProcess's multi-label selection instead of a per-query argmax, which silently dropped legitimate detections whenever a query scored above threshold on more than one class. (#1320)
• EMA no longer double-counts the last training step of every epoch, which previously bypassed ema_update_interval. (#1319)
• model.export(format="tflite") no longer hangs at the ONNX→TFLite step (an Abseil weak-symbol coalescing issue between the onnx C extension and TensorFlow). (#1323)

---

🏆 Contributors

• adenstamm (@adenstamm) — fixed missing legends on training metric plots.
• Aman Harsh (@amanharshx, LinkedIn) — fixed a TFLite export hang by preloading TensorFlow before ONNX.
• aryan kolapkar (@arubittu) — enabled strict mypy checking across the datasets and namespace modules.
• Flo (@flhoxha) — fixed YOLO-format datasets to evaluate the real test split.
• FootysHands (@ayo0la) — fixed metrics.csv history being lost across resumed training runs.
• Jesús Royeth (@JESUSROYETH) — the release's most prolific contributor this cycle: the predict() CUDA-sync fix, the matcher target-side safety-gate caching, the YOLO test-split fallback, AdamW-aware auto-batch probing, the O(N log N) confidence-threshold sweep, the keypoint/ref_wh non-finite guard, the two-stage top-k gather-order fix, the SegmentationHead skip_blocks fix, the ONNX/TFLite multi-label decode fix, and the EMA epoch-boundary double-count fix.
• jirka (@Borda, LinkedIn) — pre-commit and documentation maintenance.
• Vlad Voropaev (@voropaevv, LinkedIn) — fixed the ONNX Runtime benchmark to honor the requested CUDA device.

---

Full changelog: 1.9.2...1.9.3