Skip to content

v1.7.0: Lazy Segm data loading, GPU Augm.

Choose a tag to compare

View all tags
@Borda Borda released this 21 May 20:32
· 316 commits to develop since this release
1.7.0
e77de66
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This release fixes YOLO segmentation training OOM on large datasets, adds GPU-side augmentation for segmentation, RF_HOME weight caching, PyTorch Lightning checkpoint resume, and TFLite export — plus several developer-facing builder APIs. It also moves peft to an opt-in extra and removes the long-deprecated rfdetr.util / rfdetr.deploy import paths — see Breaking Changes below before upgrading.

✨ Highlights

Feature What changed
Seg OOM fix — lazy dataset loading YOLO seg datasets store polygon coordinates only; H × W masks rasterised per-image in __getitem__. 73 GB → tens of MB at construction.
RF_HOME — portable weight cache Bare filenames resolve relative to RF_HOME. One env var covers CI, Docker, shared storage.
PyTorch Lightning .ckpt resume Pass any PTL checkpoint as pretrain_weights. Keys auto-normalized — no manual conversion.
GPU augmentation for segmentation augmentation_backend="gpu" now augments images, boxes, and masks in sync. Previously silently ignored for seg models.
TFLite export model.export(format="tflite") via onnx2tf. FP32, FP16, INT8 (with calibration data). Seg masks decoded in TFLite inference.

🚀 Added

  • TFLite export. New model.export(format="tflite") converts through ONNX using onnx2tf. FP32 and FP16 outputs are always produced; INT8 quantization is available with a calibration image directory. Requires pip install 'rfdetr[onnx,tflite]'. (#920)

  • Kornia GPU augmentation now supports instance segmentation. Images, boxes, and per-instance masks are augmented in sync on the GPU via RFDETRDataModule.on_after_batch_transfer. Previously augmentation_backend="gpu"/"auto" was silently ignored for segmentation models. The mask buffer is [B, N_max, H, W] float32 — roughly 500 MB at B=8, N_max=50, H=W=560; use augmentation_backend="cpu" on cards with limited VRAM. (#1003)

  • Grayscale and multispectral imagery support. RF-DETR models now accept inputs with any number of channels, not just 3. The pretrained DINOv2 patch-embedding weights are automatically adapted to the specified channel count at construction time — no extra dependencies. (#180)

  • PyTorch Lightning .ckpt files accepted as pretrain_weights. Keys are auto-normalized from PTL format (state_dict with model.-prefixed keys, hyper_parameters → args) so that load_pretrain_weights, class-name extraction, and compatibility checks work without manual conversion. (#951)

  • rfdetr.from_checkpoint(path). New top-level convenience function that loads a checkpoint and infers the correct model subclass automatically — no need to know or pass the class. Equivalent to RFDETR.from_checkpoint(path). (#664)

  • skip_best_epochs parameter for RFDETR.train() and TrainConfig. The first N epochs are excluded from best-checkpoint selection and early-stopping comparison, so strong pretrained weights or resumed checkpoints can't lock in a suboptimal early score. (#1000)

  • augmentation_backend field on TrainConfig ("cpu" / "auto" / "gpu"): opt-in GPU-side augmentation via Kornia. CPU path is unchanged and remains the default. Install with pip install 'rfdetr[kornia]'. (#1003)

  • RF_HOME environment variable controls where pretrained model weights are cached (default: ~/.roboflow/models). Bare filenames passed as pretrain_weights (e.g. "rf-detr-base.pth") resolve relative to this directory; paths with a directory component are used as-is with parent directories created automatically. (#130)

  • training_config.json is now saved to the output directory after training completes. Captures the full TrainConfig, ModelConfig, effective training parameters, class names, and number of classes — useful for reproducibility and debugging predictions from older checkpoints. (#194)

  • ONNX export filenames include the model variant name (e.g. rfdetr-medium.onnx) instead of the generic inference_model.onnx. Exporting multiple variants to the same directory no longer overwrites previous exports. (#910)

  • Background images (no matching label file) are included in YOLO detection datasets as empty-detection samples instead of being silently dropped. Both detection and segmentation paths now use _LazyYoloDetectionDataset for consistent behaviour. (#915)

  • RFDETR.predict(include_source_image=...) — opt-out flag (default True) to skip storing the source image in detections.metadata["source_image"]; set to False to reduce memory use when the image is not needed for annotation. (#912)

  • model_name is now stored in checkpoint files during training so that RFDETR.from_checkpoint() can resolve the correct model class directly from the checkpoint, without requiring the caller to pass a class hint. Backward-compatible: checkpoints without model_name continue to resolve via filename matching. (#895)

  • rfdetr_version is now stored in checkpoint files during training for provenance tracking and compatibility hints. (#918)

  • dinov2_registers_windowed_small backbone is now available as a config option in ModelConfig.encoder. (#236)

  • notes parameter for train() and export() — embeds provenance metadata in .pth checkpoints (checkpoint["args"]["notes"]) and ONNX files (rfdetr_notes metadata key). Useful for tagging checkpoints with experiment descriptions or dataset versions. (#1025)

  • PretrainWeightsCompatibilityWarning is now emitted when a ModelConfig override (e.g. custom encoder or num_queries) risks breaking pretrained weight loading. Importable as from rfdetr.config import PretrainWeightsCompatibilityWarning for targeted warning suppression. (#1017)

  • TFLite inference now decodes segmentation masks into sv.Detections.mask. Mask logits are upsampled to the source image size using Pillow bilinear resampling and thresholded at zero, matching PostProcess.forward behaviour. (#1053)

Builder API surface (advanced users)

  • build_model_from_config(model_config, train_config=None, defaults=MODEL_DEFAULTS) — config-native alternative to build_model(build_namespace(mc, tc)); accepts Pydantic config objects directly. (#845)
  • build_criterion_from_config(model_config, train_config, defaults=MODEL_DEFAULTS) — config-native alternative to build_criterion_and_postprocessors(build_namespace(mc, tc)). (#845)
  • ModelDefaults dataclass and MODEL_DEFAULTS singleton — exposes the 35 hardcoded architectural constants previously buried inside build_namespace(). Customise with dataclasses.replace(MODEL_DEFAULTS, ...). (#845)
  • BuilderArgs — a @runtime_checkable typing.Protocol documenting the minimum attribute set consumed by build_model(), build_backbone(), build_transformer(), and build_criterion_and_postprocessors(). (#841)

🌱 Changed

  • convert_coco_poly_to_mask now handles RLE annotations. Both compressed (string counts) and uncompressed (int-list counts) RLE formats are decoded alongside existing polygon support. Malformed annotations now raise instead of being silently swallowed. (#897)

  • PyTorch Lightning version constraint updated to exclude known-compromised releases. If your environment pins PTL explicitly, verify it is not in the excluded range. (#1020)

⚠️ Breaking Changes

  • peft is no longer installed by default. It has moved to the [lora] and [train] optional extras. If you use LoRA fine-tuning, install with pip install 'rfdetr[lora]'. Existing rfdetr[train] installs continue to include peft. (#838)

🗑️ Deprecated

  • rfdetr.util.* and rfdetr.deploy.* import paths — both shim packages remain active in 1.7.0 and emit DeprecationWarning. Use rfdetr.utilities.* and rfdetr.export.* instead. Removal in v1.8. (#839)
  • RFDETRBase — use RFDETRNano, RFDETRSmall, RFDETRMedium, or RFDETRLarge instead. Emits FutureWarning on instantiation; scheduled for removal in v2.0. (#900)
  • RFDETRSegPreview — use RFDETRSegNano, RFDETRSegSmall, RFDETRSegMedium, or RFDETRSegLarge instead. Emits FutureWarning on instantiation; scheduled for removal in v2.0. (#900)
  • build_namespace(model_config, train_config) — use build_model_from_config, build_criterion_from_config, or _namespace_from_configs directly. Removal in v1.9. (#845)
  • load_pretrain_weights(nn_model, model_config, train_config) — the train_config positional argument is no longer used and emits DeprecationWarning. Removal in v1.9. (#845)
  • TrainConfig.group_detr, TrainConfig.ia_bce_loss, TrainConfig.segmentation_head, TrainConfig.num_select, ModelConfig.cls_loss_coef — each now emits DeprecationWarning when set on the wrong config object. Removal in v1.9. SegmentationTrainConfig users: remove the num_select override — the model config value is always used. (#841)

🔧 Fixed

  • Fixed ONNX/TRT dynamic batch inference. gen_encoder_output_proposals and Transformer.forward extracted the batch size as a Python int and passed it to torch.full, .view(N_, ...), .expand(N_, ...), and .repeat(bs, ...), baking the training batch size into the exported graph. TRT engines built with --minShapes smaller than the trace batch failed at inference with Reshape: reshaping failed. All six call sites now use ONNX-symbolic equivalents (zeros_like, -1 reshapes, expand(memory.shape[0], ...)). (#950)

  • Fixed RFDETRModelModule.on_load_checkpoint crashing with RuntimeError on resume from a different image resolution. DINOv2 positional embeddings in the checkpoint are now bicubic-interpolated to match model_config.positional_encoding_size before PyTorch Lightning applies the state dict. (#1002)

  • Fixed training failure when square_resize_div_64=False. The non-square resize pipeline (SmallestMaxSize + LongestMaxSize) did not guarantee output dimensions divisible by patch_size * num_windows, causing WindowedDinov2WithRegistersEmbeddings.forward to raise ValueError. A PadIfNeeded step is now appended in both train and val/test pipelines. (#991)

  • Fixed YOLO segmentation training out-of-memory on large datasets. supervision.DetectionDataset.from_yolo(force_masks=True) was eager-rasterising H×W boolean masks at dataset construction time (≈1 GB per 1 000 images at 1024 px). A new _LazyYoloDetectionDataset stores polygon coordinates only and defers dense mask rasterisation to __getitem__, keeping RAM proportional to annotation count. (#851)

  • Fixed _namespace.py regression where TrainConfig.num_select=300 silently overrode model-specific values of 100–200 for segmentation variants. num_select in the builder namespace now always reads from ModelConfig. (#841)

  • Fixed models/weights.py: load_pretrain_weights now correctly auto-aligns the model head when the checkpoint has fewer classes than the configured default, preventing a silent mismatch when num_classes was not explicitly set. (#845)

  • Fixed RFDETRLarge initialization showing two conflicting ValueErrors. When the deprecated-config fallback retry also fails, the fallback now re-raises the original error without chained context, so users see a single deterministic message. (#975)

  • Fixed WindowedDinov2WithRegistersEmbeddings.forward() failing silently under -O when input spatial dimensions are not divisible by patch_size * num_windows. It now raises ValueError with a clear message identifying the divisor and actual shape. (#167)

  • Fixed TFLite detection scores collapsing (all scores ~0.02 vs ~0.62 from ONNX). The GridSample ONNX node is now rewritten to Gather-based integer-index arithmetic before conversion, eliminating numerical drift from attention position sampling. (#1054)

  • Fixed class_name lookup for pretrained COCO models. COCO category IDs are sparse (1–90 with gaps for 80 classes), so flat 0-based indexing returned the wrong label. Detection now uses a coco_id → class_name mapping built from the canonical COCO_CLASSES list. Fine-tuned models use direct 0-based indexing unchanged. (#1051)

  • Fixed query scramble when loading multi-group DETR checkpoints. A num_queries/group_detr mismatch between checkpoint and model config caused queries to be silently remapped to the wrong positions, corrupting resumed training. load_pretrain_weights now correctly slices per-group query parameters and realigns the head when group counts differ. (#1019)

🏆 Contributors

A special welcome to our new contributors and a big thank you to everyone who helped with this release:

Automated contributions: @Copilot, @pre-commit-ci[bot]

Full changelog: 1.6.5...1.7.0

Contributors

Borda, valavanisleonidas, and 10 other contributors
Assets 4
Loading