RubikPi Edge Inference Study v0.1.0

Research question

On a RubikPi 3 (Qualcomm QCS6490) camera-inference pipeline, when does dynamic early exit help, and when is a fixed smaller or accelerator-backed model the better deployment option?

Answer (bounded to these models, this dataset, this image, this runtime, this methodology)

In this experiment, early exit did not improve deployment performance. On CPU, staged execution overhead outweighed the estimated compute reduction (p50 48.9 ms vs 33.6 ms for the full model). A static MobileNetV3-Small was substantially faster (p50 7.7 ms) at matching accuracy. On the Qualcomm HTP, a full INT8 ResNet-18 was sufficiently fast and accurate (p50 3.3 ms, 95.0% top-1) that a staged early-exit implementation was not justified under the measured configuration.

Results

Configuration	Top-1	Model p50	Processed throughput
Full ResNet-18, CPU FP32	95.0%	33.6 ms	28.7 fps
Early-exit ResNet-18, CPU FP32 (staged)	93.8%	48.9 ms	21.5 fps
Static MobileNetV3-S, CPU FP32	94.0%	7.7 ms	116 fps
Full ResNet-18, INT8, HTP	95.0%	3.3 ms	180 fps

• HTP use is proven by a 23x same-file delegate-vs-CPU comparison, an unsupported-precision controlled failure on the FP16 path (float models fall back to CPU — detected, not claimed as acceleration), and output/accuracy parity (benchmarks/accelerator_proof.json).
• Negative result kept visible: INT8 post-training quantization collapsed MobileNetV3-Small to 17.5% accuracy.
• Camera-path deviation: the attached CSI module is OV5647-style hardware with no driver on Qualcomm Linux 1.3, so live experiments used a laptop webcam streamed as MJPEG over the USB ADB link; the RubikPi performed decode, preprocessing, inference, routing, queueing and telemetry. This is a RubikPi inference benchmark, not a CSI-camera benchmark.
• Live behaviour: only the static model sustained the ~30 fps input; a 10-minute early-exit run held 17.5 fps processed throughput with 7,547 stale frames intentionally dropped by the bounded queue, temperatures plateauing at ~86–88 °C (frequency throttling not independently verified).
• Live room scenes are out of distribution for CIFAR-10 and are used for systems behaviour only, not accuracy claims.

Reproduction

Start at the README's Reproduce section: scripts/prepare_models.py (host, seeded) → scripts/export_models.py (parity-asserted) → board benchmarks. No model weights are committed; artifacts regenerate deterministically and SHA-256 hashes of the published runs are committed.

Limitations

See docs/limitations.md. Results are specific to this board sample, image, runtimes and models — not to early-exit systems or Qualcomm hardware in general.