Skip to content

v0.4.0 — ExecuTorch backend + RF-DETR 1.9.0

Latest

Choose a tag to compare

@olibartfast olibartfast released this 04 Aug 11:47

Third inference backend and a unified dependency-resolution layer. ExecuTorch
joins ONNX Runtime and TensorRT, running .pte programs exported by rfdetr
1.9.0+, and the export pin moves to rfdetr[onnx]==1.9.0. Dependency resolution
moves behind a single find_dependency_unified() facade covering apt, conan,
vcpkg, and provided-download strategies.

Breaking: enabling more than one inference backend is now a configure-time
error. Previously two backends configured successfully and silently used ONNX
Runtime.

RF-DETR 1.9.0 alignment + ExecuTorch backend

Upstream release: https://github.com/roboflow/rf-detr/releases/tag/1.9.0

Added

File Change
src/backends/executorch_backend.{hpp,cpp} Third inference backend, running .pte programs via executorch::extension::Module. Input/output shapes come from method_meta("forward"), so resolution auto-detection and the output-count check both work before the first inference. Input tensors are non-owning views over the caller's buffer (no per-frame copy); outputs are restricted to float32.
cmake/deps/packages/ExecuTorch.cmake Catalog entry: find_package(executorch CONFIG) against an install prefix first, FetchContent source build of v1.3.1 as fallback. Pinned to v1.3.1 to match the ExecuTorch version rfdetr[executorch]==1.9.0 resolves for the Python exporter — .pte schema compatibility across runtime versions is not guaranteed.
cmake/deps/{PackageManager,strategies/ProvidedPackageManager}.cmake New PROVIDED_FC_OPTIONS / PROVIDED_FC_SUBMODULES declaration keys so a FetchContent dependency can have build options seeded into the cache and submodules fetched recursively — both required to build ExecuTorch from source.
CMakeLists.txt USE_EXECUTORCH and EXECUTORCH_DELEGATE (xnnpack/portable) options; EXECUTORCH_ROOTDIR appended to CMAKE_PREFIX_PATH; delegate library linked so it self-registers.
deploy/export_executorch.py Export script for .pte models: --backend xnnpack/coreml/qnn, --soc required for qnn. Warns both before and after export that only xnnpack yields a .pte this project can run — -DEXECUTORCH_DELEGATE offers only xnnpack/portable, so a coreml/qnn program fails at load with Backend ... is not registered.
tests/unit/test_rfdetr_inference.cpp PreprocessFrame.ResizeIsAntialiasFree — asserts preprocess_bgr_image point-samples the bilinear 2x2 footprint rather than averaging over the source area.
docs/export.md ExecuTorch export section; native TensorRT export section.
Dockerfile INFERENCE_BACKEND=executorch variant. Builds the ExecuTorch C++ runtime from source into /opt/executorch (tag via EXECUTORCH_VERSION, default v1.3.1), applying the upstream extension_evalue_util install fix. Uses a throwaway venv with the CPU torch wheel because ExecuTorch's operator codegen does import torchgen, and installs build-essential because ExecuTorch's bundled flatcc is an ExternalProject that configures with the Unix Makefiles generator and does not inherit CMAKE_C_COMPILER. Static-linked, so the runtime stage ships no extra shared libraries and needs no GPU. The ExecuTorch step is placed before COPY . . so editing a .cpp reuses the cached layer (36s rebuild) instead of recompiling ExecuTorch (~11 min).

Changed

File Change
deploy/requirements.txt rfdetr[onnx] 1.8.3 → 1.9.0.
docs/export.md Version bumps to 1.9.0. Noted the relaxed onnxsim>=0.7.0 pin, the non-native-resolution export fix, and the in-process (polygraphy) TensorRT export path. Pinned the rfdetr[executorch] / rfdetr[tensorrt] install commands to ==1.9.0 — the extras do not constrain ExecuTorch itself, and an unpinned install can pull an exporter whose .pte the pinned v1.3.1 runtime cannot load.
README.md (accuracy) ONNX Runtime dependency section no longer claims Windows/macOS and CUDA/DirectML support: the default download is the Linux x64 CPU archive, and OnnxRuntimeBackend registers no execution provider, so even a GPU build would run on CPU. Other platforms need -DONNXRUNTIME_ROOTDIR or the conan/vcpkg coordinates. CMake requirement notes the 3.17+ needed by the ExecuTorch FetchContent fallback (GIT_SUBMODULES_RECURSE).
README.md Three-backend selection table with model formats; ExecuTorch build section, dependencies, and build options; rfdetr[executorch] / rfdetr[tensorrt] export extras; version statements to 1.9.0. Fixed the media-backend example -DUSE_TENSORRT=ON -DUSE_OPENCV=ON, which the new exactly-one-backend check turned into a configure error (USE_ONNX_RUNTIME defaults ON). Documented TensorRT's .onnx acceptance and per-backend integration-test discovery.
AGENTS.md Backend selection covers ExecuTorch; CI-coverage note extended; new Docker section.
docs/package-manager-architecture.md ExecuTorch in the catalog listing and dependency-mapping table; documented the new PROVIDED_FC_OPTIONS / PROVIDED_FC_SUBMODULES keys and the "install-prefix first, source build as fallback" pattern they enable.
docs/glossary.md Backend selection is three-way; added ExecuTorch, .pte, Delegate, and XNNPACK entries.
.github/copilot-instructions.md Backend list covers ExecuTorch.
CMakeLists.txt Behavior change: enabling more than one backend is now a configure-time error. Previously -DUSE_ONNX_RUNTIME=ON -DUSE_TENSORRT=ON configured successfully and silently used ONNX Runtime, because create_backend() resolves via an #ifdef/#elif chain. With a third backend that silent fallthrough became too easy to hit unnoticed.
tests/integration/integration_test_rfdetr_inference.cpp Skip message 1.8.3 → 1.9.0. Model discovery now probes the extensions the compiled-in backend can load instead of a hardcoded .onnx, and the skip message names the expected format. ONNX Runtime probes .onnx; ExecuTorch .pte; TensorRT .engine, .trt, then .onnx — the last kept deliberately, since TensorRTBackend::initialize() builds/caches an engine from ONNX and dropping it would lose that path from coverage. Extension is the outer loop so format preference is global: a prebuilt engine in any search directory beats an ONNX file in an earlier one, since loading an engine is instant while building one costs minutes. Also fixes a pre-existing TensorRT bug: a TRT build previously found a stray .onnx before the .engine beside it.
src/main.cpp Usage text is backend-aware: the example model extension, the backend description, and the rebuild flags are selected at compile time, instead of claiming only ONNX Runtime and TensorRT exist.
.gitignore Ignore data/test_* and data/empty_labels.txt — all six fixtures the detection and keypoint integration fixtures generate (test_image.jpg, test_labels.txt, test_output.jpg, test_kp_image.jpg, test_kp_output.jpg, empty_labels.txt), which are normally removed in TearDown but survive an interrupted run.

Why no postprocessing port or model re-export

Diffing 1.8.3...1.9.0 over the code this project mirrors:

  • export/_onnx/exporter.py (+201/-97) and models/lwdetr.py (+78/-57) are typing-only — protocols, cast, f-strings. Graph structure, output names, ordering, and opset are unchanged, so existing exported models stay compatible and no re-export is required.
  • models/postprocess.py (+63/-18) chunks mask upsampling to cut peak GPU memory (identical results — still bilinear to target size, then > 0.0) and adds upsample_masks_to_image_size, an opt-in validation-only flag. media.cpp::resize_threshold_mask already matches the default path.
  • PR #1206 confirms this project's preprocessing. Upstream predict() had been resizing with antialias enabled, drifting from training; 1.9.0 sets antialias=False to match the antialias-free bilinear (cv2.INTER_LINEAR) resize used during training. media.cpp::preprocess_bgr_image has always been exactly that, so 1.8.3's predict() was the side that disagreed, and 1.9.0 closes the gap. Locked in by PreprocessFrame.ResizeIsAntialiasFree.

End-to-end verification

The ExecuTorch backend was run end-to-end against a real .pte before this change was committed. rf-detr-nano.pth was exported at 384×384 to both .onnx and .pte (format="executorch", backend="xnnpack") from the same checkpoint with rfdetr 1.9.0, then run through inference_app on both backends:

Check Result
Detections at threshold 0.5 3 vs 3 — same classes, same order
Detections at threshold 0.05 34 vs 34 — same classes, same order
Max box delta 1e-4 px
Max score delta 1e-6
Output-order guard method_meta reports [1,300,4] boxes at index 0, [1,300,91] logits at index 1
Delegate mismatch (portable build vs xnnpack .pte) Fails with Backend XnnpackBackend is not registered, exit 1
Integration tests with RFDETR_TEST_MODEL=<.pte> 4 passed, 1 skipped (no keypoint .pte)

Regression status: ONNX Runtime 39/39, TensorRT 39/39, ExecuTorch 39/39; clang-format clean; -DWERROR=ON clean on the ExecuTorch configuration.

Upstream ExecuTorch packaging bug (affects install-prefix builds)

ExecuTorch v1.3.1 extension/evalue_util/CMakeLists.txt:27 installs its target with
DESTINATION ${CMAKE_BINARY_DIR}/lib instead of ${CMAKE_INSTALL_LIBDIR} — every other
extension uses the latter. Consequences for a cmake --installed prefix:

  • libextension_evalue_util.a is never copied into <prefix>/lib.
  • ExecuTorchTargets-release.cmake records an absolute build-tree path for that one
    target (18 of 19 targets are ${_IMPORT_PREFIX}-relative; this one is not).
  • find_package(executorch CONFIG) then hard-fails via its import-check loop as soon as
    the build tree is removed — even though this project never links that target.

Workaround when building ExecuTorch from source, applied before cmake --install:

sed -i 's|DESTINATION ${CMAKE_BINARY_DIR}/lib|DESTINATION ${CMAKE_INSTALL_LIBDIR}|' \
    extension/evalue_util/CMakeLists.txt

The FetchContent fallback path is unaffected: FetchContent_MakeAvailable consumes the
targets directly from the build tree and never runs the faulty install() rule.

package-manager abstraction

Unified package-manager abstraction layer (cmake/deps/) wrapping apt, conan,
vcpkg, and provided-download strategies behind a single find_dependency_unified()
facade. All dependencies — including GTest, Google Benchmark, stb, and font8x8 —
now route through the facade instead of ad-hoc FetchContent or manual
target_include_directories calls.

Added

File Change
cmake/deps/Deps.cmake Facade: public options (DEPS_MODE, DEPS_CONAN_DIR, DEPS_OFFLINE, DEPS_PROVIDED_DIR, DEPS_DEBUG), module includes, catalog loader, resolver construction.
cmake/deps/PackageManager.cmake Registry (deps_declare), chain-of-responsibility driver (deps_resolve), non-IMPORTED INTERFACE target builder (deps_build_targetDeps::<Name>), lockfile writer (deps_lock_writedeps.lock.json).
cmake/deps/strategies/AptPackageManager.cmake apt strategy: find_package / pkg_check_modules QUIET probe.
cmake/deps/strategies/ConanPackageManager.cmake Conan 2 strategy: toolchain mode (auto-detected) + CMakeDeps-only mode (-DDEPS_CONAN_DIR); find_package(<CONAN_FIND> CONFIG).
cmake/deps/strategies/VcpkgPackageManager.cmake vcpkg strategy: manifest mode (auto-detected via toolchain); CONFIG + MODULE mode fallback; prefers IMPORTED targets, falls back to <FIND>_LIBRARIES variables.
cmake/deps/strategies/ProvidedPackageManager.cmake Provided strategy: DOWNLOAD (pinned URL), VENDORED (third_party/), FETCHCONTENT (git clone), ROOT (user path); shared deps_provided_finalize.
cmake/deps/packages/*.cmake 10 catalog entries: OnnxRuntime, TensorRT, OpenCV, FFmpeg, SDL2, Threads, GTest, GoogleBenchmark, stb, font8x8 — each declares apt/conan/vcpkg/provided coordinates in one place.
conanfile.txt ffmpeg/6.1, sdl/2.28.5, gtest/1.12.1; CMakeDeps + CMakeToolchain generators; OpenCV alternative documented.
vcpkg.json ["ffmpeg", "sdl2", "gtest"] manifest.
.github/workflows/deps-modes.yml workflow_dispatch CI matrix testing apt/conan/vcpkg modes; conan uses gcc11 CMakeDeps-only mode with prebuilt binaries.
docs/package-manager-architecture.md 118-line architecture reference: strategies, catalog format, dependency mapping table, options, lockfile, CMakeDeps-only mode.
CMakeLists.txt DEPS_MODE cache option; all dependencies resolved via find_dependency_unified(); lockfile written at configure end.

Changed

File Change
CMakeLists.txt GTest, Google Benchmark: raw FetchContent → facade (PROVIDED_ACQUIRE=FETCHCONTENT fallback); stb, font8x8: manual target_include_directories → facade (PROVIDED_ACQUIRE=VENDORED); deps_get_rec() used for RPATH/runtime libs.
README.md New Dependency Resolution section with mode/chain table, conan CMakeDeps-only example, vcpkg manifest example, system-package notes.
AGENTS.md Brief DEPS section: backend selection, dependency resolution, DEPS_DEBUG.

Fixed

  • DEPS_OFFLINE honored for FETCHCONTENT: can_resolve returns FALSE when offline, preventing network clone attempts in air-gapped builds (Codex review P2).
  • vcpkg MODULE mode fallback: vcpkg's FFmpeg port ships a Find module (not a config file); the strategy now tries CONFIG first, then MODULE, and uses <FIND>_LIBRARIES variables when IMPORTED targets don't exist.
  • Chain order: conan;apt;provided and vcpkg;apt;provided — apt fills in system packages (Threads) even in conan/vcpkg modes.
  • Conan "version conflict" resolved: OpenCV and FFmpeg are mutually exclusive (USE_OPENCV option); each conan graph resolves independently.

Why

Decouples dependency acquisition from build logic. Previously, each dependency
had bespoke find_package/FetchContent/download code interleaved with build
targets in CMakeLists.txt. The facade centralizes all acquisition strategies
behind a single declarative interface (find_dependency_unified(Name REQUIRED)),
making it trivial to switch between apt (default, zero tooling), conan (prebuilt
binaries), and vcpkg (manifest mode) without touching build logic. The lockfile
records which handler + version resolved each dep for reproducibility.

Also includes expanded runtime-verification toolchain: ThreadSanitizer (data-race
detection) and Valgrind (memcheck correctness + callgrind/massif profiling),
wired into both CI and local CMake targets.