Skip to content

recipe(rtdetr-r18vd): add CPU object detection recipes#1190

Draft
kujin66 wants to merge 1 commit into
mainfrom
kujin66/add-pekingu-rtdetr-r18vd-coco-o365-recipe
Draft

recipe(rtdetr-r18vd): add CPU object detection recipes#1190
kujin66 wants to merge 1 commit into
mainfrom
kujin66/add-pekingu-rtdetr-r18vd-coco-o365-recipe

Conversation

@kujin66

@kujin66 kujin66 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds verified CPU recipes for PekingU/rtdetr_r18vd_coco_o365 object detection. This is an L0 recipe-only contribution with CPU fp32/fp16 build, perf, L2 PyTorch-vs-ONNX numeric comparison, analyze evidence, and explicit L3 dataset/label blocker evidence on current PR head.

Model metadata

What the model does

PekingU/rtdetr_r18vd_coco_o365 is an RT-DETR object detection model. It consumes an RGB image tensor and emits class logits plus normalized predicted boxes for COCO-style detection.

Primary user stories

  • A user supplies an image to obtain object class scores and bounding boxes for object detection (verified from config and WinML build output).

Supported tasks

Task Support surface Evidence Confidence
object-detection checkpoint, Transformers/Optimum, WinML RTDetrForObjectDetection; WinML build/perf/L2 commands below verified

Model architecture

RTDetrForObjectDetection
|-- RTDetrModel
|   |-- ResNet-style convolutional backbone
|   |-- encoder/decoder detection transformer blocks
|   `-- object query features
`-- detection heads
    |-- logits [1, 300, 80]
    `-- pred_boxes [1, 300, 4]

Validation and support evidence

Baseline

  • Base: origin/main at d9feefffccecfe47f08b7d2cfffb11fe57eb94df.
  • winml config -m PekingU/rtdetr_r18vd_coco_o365 --task object-detection succeeded and auto-detected AutoModelForObjectDetection / object-detection with pixel_values [1, 3, 640, 640] and outputs logits, pred_boxes.
  • Dynamo export retained opset 18, so the checked-in recipes pin opset 18.

Goal

  • Effort: L0 recipe-only.
  • Goal ceiling reached here: L2 on CPU for fp32 and fp16.
  • Outcome: CPU recipe coverage for exact (cpu, cpu, fp32) and (cpu, cpu, fp16) tuples.

Outcome

  • Shipped recipes:
    • examples/recipes/PekingU_rtdetr_r18vd_coco_o365/cpu/cpu/object-detection_fp32_config.json
    • examples/recipes/PekingU_rtdetr_r18vd_coco_o365/cpu/cpu/object-detection_fp16_config.json
  • Coverage: partial by target breadth; CPU fp32/fp16 verified on this host. No NPU/GPU recipes are claimed.
  • L3 is blocked by dataset label incompatibility between detection-datasets/coco and this model's label set, with exact CLI error recorded below.
  • Methodology friction: no additional skill methodology change required for this recipe-only PR.

Per-EP/device/precision results

Tier EP / Device Precision Verdict Mean p50 Throughput RAM delta Task metric
L0 CPUExecutionProvider / cpu fp32 PASS - - - - -
L0 CPUExecutionProvider / cpu fp16 PASS - - - - -
L1 CPUExecutionProvider / cpu fp32 PASS 151.52 ms 151.14 ms 6.60 samples/sec +234.8 MB total -
L1 CPUExecutionProvider / cpu fp16 PASS 313.47 ms 314.69 ms 3.19 samples/sec +279.6 MB total -
L2 CPUExecutionProvider / cpu fp32 PASS - - - - logits cosine 1.000000000 max_abs 0.000066757; boxes cosine 1.000000000 max_abs 0.000009298
L2 CPUExecutionProvider / cpu fp16 PASS with drift - - - - logits cosine 0.994308047 max_abs 3.133764744; boxes cosine 0.820119791 max_abs 0.974431098
L3 CPUExecutionProvider / cpu fp32 CLI-BLOCKED-DATA - - - - Dataset label motorcycle not in model label2id (motorbike)
L3 CPUExecutionProvider / cpu fp16 CLI-BLOCKED-DATA - - - - Same dataset/label blocker as fp32; artifact already passed L0/L1/L2

Build evidence:

  • fp32: Build complete in 137.3s; final artifact model.onnx; input pixel_values [1, 3, 640, 640]; outputs logits [1, 300, 80], pred_boxes [1, 300, 4].
  • fp16: Build complete in 53.3s; FP16 stage emitted 38.8 MB artifact; perf reported Model Precision: fp16.

L3 eval attempt evidence:

  • winml eval -m temp\verify_rtdetr_fp32\model.onnx --model-id PekingU/rtdetr_r18vd_coco_o365 --task object-detection --ep cpu --device cpu downloaded/generated detection-datasets/coco, then failed because dataset label motorcycle is not in model label2id; this model uses motorbike.

Quality gates:

  • GitHub checks on PR head c973471005ebb6f6cfbd980754574bb963602002: lint, Analyze (Python), test (models), test (commands), test (analyze), test (optim), test (remaining), CodeQL, and license/cla all completed successfully.

Delta

  • Recipe-only change, based on the existing object-detection recipe pattern and RT-DETR auto-config output.
  • The production recipe README is untouched.

Analyze summary

Static rule analysis completed for fp32. fp16 standalone analyze was attempted and failed in ONNX type inference; this is recorded as analyzer coverage blocker, not runtime failure.

Component-level summary

Artifact Architecture coverage Mapping Actionable EP findings
fp32 RT-DETR convolutional backbone; encoder/decoder detector blocks; logits/box heads 697/697 ONNX nodes tagged during export QNN partial: AveragePool, Erf, GatherElements, Reshape; GridSample/Tile/TopK unknown across accelerator rule sets
fp16 RT-DETR convolutional backbone; encoder/decoder detector blocks; logits/box heads L0/L1/L2 verified; standalone analyze blocked by type inference Analysis failed: [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10)

Op-level summary

Artifact Graph Dominant ops EP roll-up
fp32 657 ops / 30 types Add 132; Reshape 85; Mul 70; Conv 69; MatMul 67 TensorRT/OpenVINO 616 supported / 41 unknown; QNN 528 supported / 88 partial / 41 unknown
fp16 664 ops / 31 types before analyzer failure Add/Reshape/Mul/Conv/MatMul dominate by model family; full EP classification unavailable ONNX type inference blocker before EP classification

Rule-less EPs (CUDA, MIGraphX, DML): all operators unknown in static rules where analysis reaches EP classification.

Reproduce commands

$OUT='temp/verify_rtdetr_fp32'
uv run winml build -c examples\recipes\PekingU_rtdetr_r18vd_coco_o365\cpu\cpu\object-detection_fp32_config.json -m PekingU/rtdetr_r18vd_coco_o365 -o $OUT --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild
uv run winml perf -m $OUT\model.onnx --ep cpu --device cpu

$OUT='temp/verify_rtdetr_fp16'
uv run winml build -c examples\recipes\PekingU_rtdetr_r18vd_coco_o365\cpu\cpu\object-detection_fp16_config.json -m PekingU/rtdetr_r18vd_coco_o365 -o $OUT --ep cpu --device cpu --precision fp16 --no-analyze --no-optimize --no-compile --rebuild
uv run winml perf -m $OUT\model.onnx --ep cpu --device cpu

python temp\compare_rtdetr_l2.py
uv run winml eval -m temp\verify_rtdetr_fp32\model.onnx --model-id PekingU/rtdetr_r18vd_coco_o365 --task object-detection --ep cpu --device cpu

$env:WINMLCLI_RULES_DIR='<rules-dir>'
uv run winml analyze --model temp\verify_rtdetr_fp32\model.onnx --ep all --output temp\verify_rtdetr_fp32\analyze_all.json
uv run winml analyze --model temp\verify_rtdetr_fp16\model.onnx --ep all --output temp\verify_rtdetr_fp16\analyze_all.json
Remove-Item Env:WINMLCLI_RULES_DIR

@kujin66 kujin66 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Jul 24, 2026
@kujin66

kujin66 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

APPROVE (skill reviewer verdict)

Reviewed PR #1190 against the adding-model-support reviewer checklist.

  • Shipment state: draft PR, model-scale-by-skill label present, head c973471005ebb6f6cfbd980754574bb963602002.
  • Diff scope: recipe-only Lane B, exactly examples/recipes/PekingU_rtdetr_r18vd_coco_o365/cpu/cpu/object-detection_fp32_config.json and _fp16_config.json; examples/recipes/README.md is untouched.
  • Required PR hierarchy: Summary, Model metadata, and Validation/support evidence are present with Baseline, Goal, Outcome, per-EP/device/precision results, Delta, Analyze summary, and Reproduce commands.
  • Verification evidence: CPU fp32/fp16 L0 build and L1 perf are recorded; L2 PyTorch-vs-ONNX comparison now recorded for both fp32 and fp16. fp16 L2 has large numeric drift but the command completed, so this is recorded as PASS with drift rather than a crash failure.
  • L3 evidence: CPU eval was attempted and blocked by dataset/model label mismatch (motorcycle vs model label motorbike), with the exact CLI error summarized in the PR body.
  • Analyze evidence: fp32 static analyze summary is recorded. fp16 standalone analyze was attempted and failed before EP classification with [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10), recorded as analyzer-specific blocker, not runtime failure.
  • GitHub checks on reviewed SHA: Analyze (Python), lint, test (analyze), test (commands), test (models), test (optim), test (remaining), CodeQL, and license/cla all completed successfully.
  • Review threads/comments: REST line comments empty; issue comments empty before this verdict; GraphQL reviewThreads empty.

No requested changes. Leave the PR in draft state per skill workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant