YOLO-Master-EsMoE-N — Edge Inference Pre-built Binaries

🚀 YOLO-Master-EsMoE-N — Edge Inference Bundles (v0.1-edge)

Prebuilt, self-contained C++ edge-inference bundles for YOLO-Master-EsMoE-N
(trained on VisDrone, 10 classes). A universal CLI auto-detects backend, class names, and input
size from the model. The bundles now span the full edge matrix — CPU (ONNXRuntime / NCNN on
Linux & Windows) and embedded GPU (native TensorRT on Jetson Orin).

Verified vs the PyTorch original on 548 VisDrone val images: mAP50 −0.10%, mAP50-95 −0.06% (< 0.5%).

Downloads

Bundle	Target	Install needed	Size
yolomaster_edge-linux-x64.tar.gz	any Ubuntu 22.04+ x86_64 (glibc ≥ 2.35)	none — $ORIGIN-rpath'd, libs bundled	35 MB
yolomaster_edge-win-x64.zip	any Windows 10/11 x64	none — MSVC runtime bundled	52 MB
yolomaster_edge-jetson-orin-jp7.tar.gz	any Jetson Orin (Nano/NX/AGX, sm87) on JetPack 7	none — OpenCV bundled; uses stock JetPack TensorRT + CUDA	13 MB

• x86 bundles are fully standalone (Linux runs with no LD_LIBRARY_PATH; Windows bundles its own VC runtime) and ship both ONNX and NCNN models under models/.
• Jetson bundle bundles OpenCV ($ORIGIN/lib rpath) and ships the FP32 ONNX; TensorRT + CUDA are not bundled — they ship with JetPack 7 and are version-matched on each device. You build the FP16 engine once, on-device.

Run

Linux (x86)

tar xzf yolomaster_edge-linux-x64.tar.gz && cd linux-x64
./yolomaster_edge --model models/esmoe_n_visdrone_sim.onnx --source <img|dir> --out out  # onnx
./yolomaster_edge --model models/esmoe_n_visdrone_ncnn --source <img|dir> --out out       # ncnn

Windows

yolomaster_edge.exe --model models\esmoe_n_visdrone_sim.onnx --source your_own_example.jpg --out out

Jetson Orin (JetPack 7)

tar xzf yolomaster_edge-jetson-orin-jp7.tar.gz && cd yolomaster_edge-jetson-orin-jp7
./build_engine.sh          # once per device — builds models/esmoe_n_fp16.engine (~10–15 min)
./yolomaster_edge --model models/esmoe_n_fp16.engine \
    --source <img|dir> --classes visdrone --conf 0.25 --out out

Backend, class names, and imgsz auto-detect from the model. Flags: --conf --iou --imgsz --multi-label --no-save --quiet (--help for all).

Example

Input	Output

Benchmark (VisDrone val, per-frame inference)

Platform	Backend	infer	FPS
💻 Windows 11 CPU	ONNX (ORT)	37.6 ms	25.4
💻 Windows 11 CPU	NCNN	80.1 ms	12.2
🤖 Jetson Orin Nano Super 4GB	TensorRT FP16 (C++)	27.8 ms	35.7
Linux GPU (H200)	ONNX CUDA (C++)	7.8 ms	~128

🏎️💰 Real-time FP16 inference on a 10W, $250 edge device!

Jetson Orin Nano 4 GB detail (FP16 engine, batch-1, 640×640): GPU compute 27.76 ms
(min 27.38 / p99 27.88); end-to-end 28.7 ms (H2D 0.90 + compute 27.76 + D2H 0.05); C++ runner
~30.8 ms/frame → ~33 FPS streaming. Cold-start first inference ~160 ms.

Jetson accuracy (on-device, VisDrone val, 548 images): FP16 engine mAP50 0.3488 / mAP50-95 0.2029
(−0.46% / −0.34% vs FP32 0.3504 / 0.2036) — near-lossless, matching the desktop ONNX reference to within
0.2% mAP points. Reminder: FP16 beats INT8 on this MoE + area-attention architecture — INT8's attention/head
layers fall back to FP32, making QDQ INT8 both slower (45.4 ms) and −2.9 mAP points, while uncalibrated
--int8 collapses completely (mAP50 0.128). For such kinds of mixed detectors, ship FP16.

Notes

• x86_64 CPU tier: Linux floor is glibc 2.35 (Ubuntu 22.04+); older distros may need a build on an older glibc.
• Jetson engine is per-device. TensorRT ties the .engine to the specific GPU + TRT version — do not copy esmoe_n_fp16.engine between machines; re-run build_engine.sh on each device. One bundle serves the whole Orin family (Nano/NX/AGX).
• 4 GB Nano: the engine build is memory-hungry — build_engine.sh uses --builderOptimizationLevel=3 (sidesteps a TensorRT 10.16 FP16 KTM build bug on Orin) and auto-adds an 8 GB swapfile if none exists. Inference itself needs only ~20 MB and runs comfortably. NX/AGX with more RAM simply build faster.
• JetPack 7 required (CUDA 13.x, TensorRT 10.x, Ubuntu 24.04 aarch64). On older JetPack (CUDA 12 / TRT 8–9) the ABI differs — require to rebuild from source (jetson/21_build_trt_runner.sh).
• Build from source: see README.md; full Jetson build/repro details and the reproduced TensorRT-on-Orin gotchas are in jetson/DEPLOYMENT_LOG.md.