Skip to content

v0.0.1 - initial release

Latest
Latest

Choose a tag to compare

View all tags
@farukalamai farukalamai released this 27 Jun 17:01
v0.0.1
63280ce
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

We are excited to announce the initial public release of RF-DETR C++, a production-grade TensorRT inference library for RF-DETR, Roboflow's transformer-based real-time object detection model built on a DINOv2 backbone.

RF-DETR C++ brings zero-Python, GPU-accelerated inference to C++ applications. The entire pipeline — preprocessing, inference, and postprocessing — runs in C++ with CUDA, delivering sub-3ms latency on modern NVIDIA GPUs.

Features

Object Detection

  • RFDetrDetector with detect() and detect_batch() APIs
  • Fused resize and normalize CUDA preprocessing kernel with async H2D transfers via pinned memory
  • CUDA Graph capture for minimal dispatch overhead on repeated inference calls
  • Supports all RF-DETR variants: nano, small, medium, base, large

Instance Segmentation

  • RFDetrSegmenter with segment() and segment_batch() APIs
  • GPU mask decode kernel — bilinear upsample and threshold parallelized across all detections simultaneously
  • Per-detection CV_8UC1 masks at original image resolution
  • Supports segmentation variants: seg-nano, seg-small, seg-medium, seg-large, seg-xlarge, seg-2xlarge

Precision Support

  • FP32 — full precision, default
  • FP16 — ~25% faster than FP32, pre-converted ONNX weights for TRT 11
  • INT8 — lowest memory footprint, QDQ ONNX path for TRT 11+, calibration cache for TRT < 11

Memory and Resource Management

  • RAII CUDA wrappers replacing raw pointer management throughout the library
  • All CUDA handles owned and destroyed automatically

C ABI

  • Full C API (librfdetr_c.so) for FFI bindings from Python, Rust, Go, or any language with a C FFI

CLI Tools

  • rfdetr_build — convert ONNX to TensorRT engine (FP32, FP16, INT8, dynamic batch)
  • rfdetr_detect — run detection on a single image
  • rfdetr_seg — run instance segmentation on a single image
  • rfdetr_video — run detection on a video file or live camera stream
  • rfdetr_batch — run detection on multiple images in a single engine call
  • rfdetr_bench — full latency and throughput benchmark with JSON output
  • rfdetr_inspect — print engine I/O bindings, shapes, and sidecar metadata
  • rfdetr_smoke — load an engine, run a zero-input forward pass, verify outputs

Benchmark

NVIDIA RTX 5070 Ti, Batch 1, 500 iterations.

Task Variant Precision FPS Avg Latency GPU Memory
Detection nano FP16 500 2.001 ms 768 MB
Detection nano INT8 482 2.073 ms 684 MB
Segmentation seg-nano FP16 166 6.014 ms 735 MB
Segmentation seg-nano INT8 126 7.895 ms 687 MB

Requirements

Dependency Version
NVIDIA GPU CC >= 8.0
CUDA Toolkit >= 12.0
TensorRT >= 10.0
OpenCV >= 4.5
CMake >= 3.20
C++ compiler C++17

Getting Started

git clone https://github.com/infracv/rf-detr-cpp.git
cd rf-detr-cpp
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120
cmake --build build -j$(nproc)

See the README for the full setup guide including ONNX export and engine conversion.

Assets 2
Loading