Out-of-the-Box AI Model Deployment Pipelines and Full-Link Conversion Tutorials Based on D-Robotics BPU
English | 简体中文
This repository is the official collection of BPU model examples and tools (Model Zoo) provided by D-Robotics. It is oriented towards AI model deployment and application development on BPU (Brain Processing Unit), helping developers to quickly get started with BPU and fast-track model inference workflows.
The samples provide model preparation, conversion guides and board-side inference programs, with preprocessing, BPU execution, postprocessing and result output. Each sample documents its supported boards and available conversion steps.
- 🚀 Quick BPU Adoption: Provides inference programs with model download and run commands.
- 🧩 Complete End-to-End Examples: Covers the entire process from algorithm export and fixed-point quantization to efficient on-board execution (
.bin/.hbm). Includes model loading, preprocessing, BPU inference execution, post-processing, and result visualization. - 📐 Standardized Design & Documentation: Provides unified directory structures and sample code specifications, supporting Python (
hbm_runtime) and C/C++ interfaces for easy understanding, secondary development, and reduced integration/maintenance costs. - 🌐 Full Scenario Coverage: Covers classification, detection, segmentation, depth estimation, OCR, speech, and multi-modal models.
The samples/ tree is the unified source for all maintained boards. Choose the runtime and model for your board inside each sample: Python entries select the target with the --target or --platform option their README documents, native C++ entries take the arguments their README documents, and each README states which boards its published artifacts support. Platform artifacts are recorded in the manifests under docs/release/x5/ and docs/release/s/. The default branch main is the delivery line and develop is the unified development branch; the preserved rdk_x5 and rdk_s branches are historical per-board delivery lines, and RDK X3 material is archived on rdk_x3.
| Target Hardware | Where | Description |
|---|---|---|
| RDK X5 | samples/ tree, model target x5 |
Recommended system version: RDK OS >= 3.5.0, based on Ubuntu 22.04 aarch64 and TROS-Humble. |
| RDK S100 | samples/ tree, model target s100 |
Nash-E artifacts published under docs/release/s/. |
| RDK S100P | samples/ tree, model target s100p |
Nash-M artifacts published under docs/release/s/. |
| RDK S600 | samples/ tree, model target s600 |
Nash-P artifacts published under docs/release/s/. |
| Delivery / development | main (default) / develop |
Delivery branch and unified development branch. |
| Historical per-board delivery lines | rdk_x5 / rdk_s |
Preserved board-specific release branches. |
| RDK X3 (archive) | rdk_x3 |
Historical archive only, not an adaptation target. |
Browse the online model catalog →
The online catalog presents released models as searchable cards with model assets, measured performance and accuracy results, and their measurement conditions.
Click to expand project directory architecture
rdk_model_zoo/
|-- samples/
| |-- llm/ # On-device text generation (native C++)
| | |-- gemma4-e2b/ # VLM chat runtime (native C++)
| | `-- minicpm5-2b/ # Text generation (native C++)
| |-- robotics/
| | `-- himloco/ # Unitree Go2 locomotion policy
| |-- speech/
| | |-- asr/ # Speech recognition
| | |-- kws/ # Keyword spotting
| | |-- paraformer/ # Paraformer speech recognition
| |-- vla/ # Upstream VLA integrations (ACT / Pi0, pinned submodules)
| |-- vision/
| | |-- 3dresnet/ # Video action classification
| | |-- bytetrack/ # Multi-object tracking
| | |-- clip/ # Image-text multimodal matching
| | |-- convnext/ # Image classification
| | |-- depth_anything_v2/ # Monocular depth estimation
| | |-- diffusiondrive/ # End-to-end driving planning
| | |-- dinov2/ # Vision feature encoder
| | |-- edgenext/ # Image classification
| | |-- efficient_sam/ # Promptable image segmentation
| | |-- efficientformer/ # Image classification
| | |-- efficientformerv2/ # Image classification
| | |-- efficientnet/ # Image classification
| | |-- efficientvit/ # Image classification
| | |-- fasterrcnn/ # COCO detection and person tracking
| | |-- fasternet/ # Image classification
| | |-- fastvit/ # Image classification
| | |-- fcos/ # Object detection
| | |-- googlenet/ # Image classification
| | |-- hgnetv2/ # Image classification
| | |-- hobot_mot/ # Multi-object tracking (CPU IOU 2.0)
| | |-- lanenet/ # Lane detection
| | |-- lite_avatar/ # Audio-driven talking avatar
| | |-- lprnet/ # License plate recognition
| | |-- mobile_sam/ # Promptable image segmentation
| | |-- mobilenetv1/ # Image classification
| | |-- mobilenetv2/ # Image classification
| | |-- mobilenetv3/ # Image classification
| | |-- mobilenetv4/ # Image classification
| | |-- mobileone/ # Image classification
| | |-- modnet/ # Image matting
| | |-- paddle_ocr/ # OCR text detection and recognition
| | |-- pointnet/ # Point cloud part segmentation
| | |-- pp_liteseg/ # Semantic segmentation
| | |-- repghost/ # Image classification
| | |-- repvgg/ # Image classification
| | |-- repvit/ # Image classification
| | |-- resnet/ # Image classification
| | |-- resnext/ # Image classification
| | |-- siglip/ # Vision-language encoder
| | |-- ultralytics_yolo/ # Detection, segmentation, pose, classification (YOLOv5u-YOLO26)
| | |-- unet/ # Semantic segmentation
| | |-- unetmobilenet/ # Semantic segmentation
| | |-- vargconvnet/ # Image classification
| | |-- vit/ # Image classification
| | |-- yolo26_depth/ # Monocular depth estimation
| | |-- yoloe/ # Prompt-free instance segmentation
| | |-- yolov5/ # Object detection
| `-- yoloworld/ # Open-vocabulary object detection
|-- datasets/ # Sample datasets and download scripts
|-- docs/ # Project guidelines and reference documentation
|-- skills/ # RDK Model Zoo Agent Skills
`-- utils/ # Shared C++ / Python utilities- Check system version: Ensure the target board is running
RDK OS >= 3.5.0. - Connect hardware: Ensure your RDK board is powered and network-connected. SSH or VSCode Remote SSH is recommended.
- Read the model README first: Always open the target directory
README.mdbefore running commands. - Run the Ultralytics YOLO11x detection sample:
cd samples/vision/ultralytics_yolo/model
bash download_model.sh --platform x5 --family yolo11 --task detect --model-size x
cd ../runtime/python
python3 main.py \
--task detect \
--platform x5 \
--family yolo11 \
--model-size x \
--test-img ../../test_data/bus.jpg \
--img-save-path ../../test_data/inference_yolo11x.jpgInference Result:
The table lists each model family with its platforms and available models; choose the Python or C++ runtime and a compatible model artifact in each sample's support table.
| Category | Model Name | Model Path | Supported Platform | Details |
|---|---|---|---|---|
| Video Action Classification | 3D ResNet | samples/vision/3dresnet |
RDK S100 | Details |
| Image Classification | ConvNeXt | samples/vision/convnext |
RDK X5 | Details |
| Image Classification | EdgeNeXt | samples/vision/edgenext |
RDK X5 | Details |
| Image Classification | EfficientFormer | samples/vision/efficientformer |
RDK X5 | Details |
| Image Classification | EfficientFormerV2 | samples/vision/efficientformerv2 |
RDK X5 | Details |
| Image Classification | EfficientNet | samples/vision/efficientnet |
RDK X5 / RDK S100 / RDK S600 | Details |
| Image Classification | EfficientViT | samples/vision/efficientvit |
RDK X5 | Details |
| Image Classification | FasterNet | samples/vision/fasternet |
RDK X5 | Details |
| Image Classification | FastViT | samples/vision/fastvit |
RDK X5 | Details |
| Image Classification | GoogLeNet | samples/vision/googlenet |
RDK X5 | Details |
| Image Classification | HGNetV2 | samples/vision/hgnetv2 |
RDK X5 | Details |
| Image Classification | MobileNetV1 | samples/vision/mobilenetv1 |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Image Classification | MobileNetV2 | samples/vision/mobilenetv2 |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Image Classification | MobileNetV3 | samples/vision/mobilenetv3 |
RDK S100 / RDK S100P / RDK S600 | Details |
| Image Classification | MobileNetV4 | samples/vision/mobilenetv4 |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Image Classification | MobileOne | samples/vision/mobileone |
RDK X5 | Details |
| Image Classification | RepGhost | samples/vision/repghost |
RDK X5 | Details |
| Image Classification | RepVGG | samples/vision/repvgg |
RDK X5 | Details |
| Image Classification | RepViT | samples/vision/repvit |
RDK X5 | Details |
| Image Classification | ResNet | samples/vision/resnet |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Image Classification | ResNeXt | samples/vision/resnext |
RDK X5 | Details |
| Image Classification | VargConvNet | samples/vision/vargconvnet |
RDK X5 | Details |
| Image Classification | ViT | samples/vision/vit |
RDK S100 | Details |
| Multi-Object Tracking | ByteTrack | samples/vision/bytetrack |
RDK X5 / RDK S100 / RDK S100P (local detector model) / RDK S600 | Details |
| Multi-Object Tracking | HobotMot (IOU 2.0) | samples/vision/hobot_mot |
RDK X5 / RDK S100 / RDK S100P / RDK S600 (CPU tracker over detector results) | Details |
| Object Detection / Person Tracking | Faster R-CNN MobileNetV3-Large-320-FPN | samples/vision/fasterrcnn |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Promptable Image Segmentation | EfficientSAM-Tiny | samples/vision/efficient_sam |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Promptable Image Segmentation | MobileSAM | samples/vision/mobile_sam |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Monocular Depth Estimation | Depth Anything V2 | samples/vision/depth_anything_v2 |
RDK S100 | Details |
| Monocular Depth Estimation | YOLO26 Depth | samples/vision/yolo26_depth |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Semantic Segmentation | PP-LiteSeg | samples/vision/pp_liteseg |
RDK X5 | Details |
| Semantic Segmentation | UNet ResNet Family | samples/vision/unet |
RDK X5 | Details |
| Semantic Segmentation | UNetMobileNet | samples/vision/unetmobilenet |
RDK S100 / RDK S600 | Details |
| Lane Detection | LaneNet | samples/vision/lanenet |
RDK S100 | Details |
| Object Detection | FCOS | samples/vision/fcos |
RDK X5 | Details |
| Object Detection | YOLOv5 | samples/vision/yolov5 |
RDK X5 / RDK S100 / RDK S600 | Details |
| Object Detection / Instance Segmentation / Pose Estimation / Image Classification / Oriented Bounding Boxes | Ultralytics YOLO (YOLOv5u / YOLOv8 / YOLOv9 / YOLOv10 / YOLO11 / YOLO12 / YOLO13 / YOLO26) |
samples/vision/ultralytics_yolo |
RDK X5 / RDK S100 / RDK S100P / RDK S600 | Details |
| Prompt-Free Instance Segmentation | YOLOE | samples/vision/yoloe |
RDK X5 / RDK S100 / RDK S100P | Details |
| Open-Vocabulary Object Detection | YOLOWorld | samples/vision/yoloworld |
RDK X5 | Details |
| Point Cloud Segmentation | PointNet | samples/vision/pointnet |
RDK S100 | Details |
| Image Matting | MODNet | samples/vision/modnet |
RDK X5 | Details |
| OCR Text Detection and Recognition | PaddleOCR | samples/vision/paddle_ocr |
RDK X5 / RDK S100 / RDK S600 (C++; matching models required) | Details |
| License Plate Recognition | LPRNet | samples/vision/lprnet |
RDK X5 | Details |
| Image-Text Multimodal Matching | CLIP | samples/vision/clip |
RDK X5 | Details |
| Vision Feature Encoder | DINOv2 | samples/vision/dinov2 |
RDK S100 / RDK S100P / RDK S600 | Details |
| Vision-Language Encoder | SigLIP | samples/vision/siglip |
RDK S100 / RDK S100P | Details |
| Audio-Driven Talking Avatar | LiteAvatar | samples/vision/lite_avatar |
RDK S100 / RDK S100P / RDK S600 | Details |
| Speech Recognition | ASR (Wav2Vec2) | samples/speech/asr |
RDK S100 / RDK S600 | Details |
| Keyword Spotting | KWS | samples/speech/kws |
RDK S100 | Details |
| Speech Recognition | Paraformer | samples/speech/paraformer |
RDK S100 | Details |
| Planning / End-to-End Driving | DiffusionDrive | samples/vision/diffusiondrive |
RDK S100P / RDK S600 | Details |
| Legged Locomotion | HIMLoco (Unitree Go2) | samples/robotics/himloco |
RDK X5 | Details |
| Vision-Language Model | Gemma4-E2B (native C++) | samples/llm/gemma4-e2b |
RDK S100 (user-supplied models) / RDK S100P / RDK S600 | Details |
| Text Generation | MiniCPM5-2B (native C++) | samples/llm/minicpm5-2b |
RDK S100 / RDK S100P / RDK S600 | Details |
Upstream VLA integrations (ACT and Pi0) are additionally available under samples/vla/ as pinned Git submodules; see the VLA overview for their integration scope.
- Model Docs: Each model's top-level
README.mdprovides an overview and run guide. - Source Reference: For code-level interface details, see Source Documentation.
- Guidelines: To contribute or develop, please read the Model Zoo Repository Guidelines.
- Runtime Standards: Runtime code standard (model interfaces and data flow) and README standard.
- Architecture: The readable model-example pattern is described in Model examples.
- Release & Manifests: Board identities and model manifests are listed under docs/release.
- Toolchain Manuals:
- Developer Forum: D-Robotics Developer Community
- User Manual: RDK User Manual
1. Model accuracy doesn't meet expectations?
- Ensure OpenExplorer Docker and board-side
libdnn.soversions are up-to-date. - Check if model export followed the structure adjustments/operator replacements required in the model's README.
- Verify cosine similarity of each output node is >= 0.999 (minimum 0.99) during quantization validation.
2. Inference speed doesn't meet expectations?
- Python API performance is lower than C/C++. For maximum performance, use C/C++.
- Benchmark data (pure forward) excludes pre/post-processing. Models with NV12 input usually achieve peak BPU throughput.
- Ensure CPU/BPU frequency is locked to maximum.
- Check for other resource-heavy processes.
3. How to fix quantization precision loss?
- Refer to the PTQ accuracy debugging section in the platform documentation.
- If INT8 loss is severe due to model characteristics, consider Mixed Precision or QAT (Quantization-Aware Training).
4. Error "Can't reshape 1354752 in (1,3,640,640)"?
The conversion settings no longer match the ONNX input size. Update the input resolution in the sample's conversion/ guide settings to match your ONNX model, delete old calibration data and re-run the calibration script.
5. mAP accuracy is lower than official results (e.g., Ultralytics)?
Compare results with the same dataset, input resolution, preprocessing, thresholds and evaluation script. Check the floating-point model first, then compare the quantized model using the same inputs. Include RGB/NV12 conversion in the preprocessing comparison.
6. Does the model use CPU during inference?
Yes. Non-quantizable or BPU-unsupported operators fallback to CPU. Even for pure BPU models, input/output quantization/dequantization nodes are executed by the CPU.
We warmly welcome contributions! Please raise an issue on GitHub Issues or discuss on the Developer Community.
This project is licensed under the Apache License 2.0 agreement.