IMX500 MCU SDK

Bring sensor-side AI vision to MCU and low-cost SoC products.

imx500-mcu-sdk helps MCU applications control Arducam IMX500 camera modules, start inference streams, and consume AI metadata or tensors over a portable I2C + SPI interface. Use it when your product needs vision events from the sensor, not a full Linux vision pipeline.

Choose Your Mission

Start with one small win. Each mission gives you a visible checkpoint and unlocks the next IMX500 product path.

Mission	Start with	Success checkpoint	Next unlock
Validate the camera over USB	Python USB tools	USB bridge detected, SDK imx500_open() completes, one metadata frame is received, optional JPEG preview is saved	USB3 UVC deployment path / B0566
Run a visible AI demo	ESP32-P4 example	LCD preview is visible and the serial log prints parsed AI output	MCU integration path
Read AI metadata on an MCU	Pico 2 serial stream example	MCU receives an IMX500 metadata frame and forwards or parses it into an event	SPI metadata productization path
Prototype with MicroPython	Pico MicroPython metadata parse example	Pico imports imx500_mcu_sdk, parses metadata, and prints SSD MobileNet detections or UART product frames	MicroPython user module
Bring up Nordic MicroPython	nRF52840 DK MicroPython SPI receive example	nRF52840 DK imports imx500_mcu_sdk, opens IMX500 over TWI1, and reads one metadata frame over SPIM3	MicroPython user module
Test a model	Model validation mission	Known model or real hardware output produces parsed metadata	Application pack / model conversion
Train your own model	Custom model training guide	Trained model is converted, packaged into .fpk + network_info.txt, flashed to B0642, and parsed with custom post-processing	Model validation to production
Port the SDK to your board	Integration guide: 中文, English, 日本語	Your platform callbacks can probe the module, start the stream, and read metadata	Custom MCU product path
Move toward production	Production checklist	Interface, model, optics, enclosure, and factory test flow are confirmed	Design-in, customization, and volume supply

Need the API surface while building? Use the Doxygen API docs.

Quick USB Validation

Demo Video

If you just received a B0642 module, validate the camera from a PC before wiring an MCU. The optional Python host tools can exercise the USB bridge, imx500_mcu_sdk.imx500_open(), metadata reads, JPEG preview, and model flashing flows.

Before connecting the module to the host PC, hold down the module MODE button, then plug in the host-side USB cable. Keep MODE button pressed until the USB bridge is detected by the host.

git clone --recursive https://github.com/ArduCAM/imx500-mcu-sdk.git
cd imx500-mcu-sdk
git submodule update --init --recursive
cd python_bindings
python3 -m pip install pybind11
python3 -m pip install -e . --no-build-isolation

Run the bundled YOLOv8 object-detection preview. The tool loads the selected model and matching network_info.txt directly for this validation session, so you do not need to flash a model first:

PYTHONPATH=python python3 tools/imx500_yolo_output_tensor_tasks.py \
  --task object_detection --preview

Press q or Esc in the OpenCV window to stop the preview. You passed this checkpoint when the tool connects to the USB bridge, imx500_open() succeeds, and the live preview shows detections.

The tool supports four bundled YOLO tasks:

--task value	Bundled model	Preview result
classification	YOLOv8n-cls	Top image classes and confidence scores.
object_detection	YOLOv8n	Object boxes, COCO class labels, and confidence scores.
pose_estimation	YOLOv8n-pose	Person boxes plus 17 COCO body keypoints and skeleton links.
segmentation	YOLOv8n-seg	Object boxes, COCO class labels, and per-instance masks.

Next unlocks:

• Want PC/Linux plug-and-play deployment? Continue toward the USB3 UVC path.
• Want a visual MCU demo? Continue with the ESP32-P4 example.
• Want to test a model first? Continue with the model validation mission.
• Want to train your own model? Continue with the custom model training guide.
• Want Linux-hosted product packaging? Continue with the MIPI / Raspberry Pi / CM5 path.
• Want low-power event output? Continue with the SPI metadata path.

Performance Benchmark

The observed frame rates for the benchmarked models are summarized below. All values are frames per second (FPS).

Model	Task	MIPI image	SPI: JPEG input tensor + output tensor	SPI: output tensor only
YOLOv8n	Object detection	30	3.5	15
YOLOv8n-pose	Pose estimation	30	3.5	10
YOLOv8n-cls	Image classification	30	2.13	30
YOLOv8n-seg	Instance segmentation	30	3.3	7.5
SSD MobileNet	Object detection	30	3.3	15
HigherHRNet	Pose estimation	30	2.45	4
MobileNetV2	Image classification	30	6.5	30
DeepLabV3+	Semantic segmentation	30	2	4.46

See the benchmark documentation for metric definitions and a link to the reproducible Pico 2 SPI benchmark.

Train Your Own Model

If the bundled models do not match your scene, start with the custom model training guide. It walks through the YOLOv8n example path from training, IMX500 conversion and quantization, Raspberry Pi or online packaging, .rpk to .fpk extraction, USB flashing to B0642, and the custom post-processing work required to turn output tensors into product events.

After the custom model produces readable metadata, use the model validation to production path to decide whether the result is ready for USB, MIPI/Linux, or SPI/MCU productization.

What You Can Build

• People counting and occupancy sensing
• Object, package, or shelf-counting devices
• Safety-zone and region-of-interest monitoring
• Smart HMI devices that react to nearby people or objects
• Low-bandwidth IoT vision endpoints that report events instead of video
• MCU-based products that need IMX500 inference without a Linux host

How It Works

MCU application
    |
    |  C/C++ SDK API
    v
IMX500 MCU SDK
    |
    |  I2C control, SPI metadata / tensor payload
    v
Arducam IMX500 camera module
    |
    |  MIPI image stream, sensor-side AI output
    v
Display, host processor, event logic, or product firmware

The SDK provides:

• I2C and SPI callback registration for platform portability
• Firmware and network-info loading support
• Stream control and runtime state queries
• Metadata readout over SPI
• Metadata parsing into network and tensor descriptors
• Optional SPI forwarding modes for validation workflows
• A platform adapter pattern for bringing up new MCU boards

Supported Platforms

Platform	Status	Entry point
ESP32-P4 Function EV Board	Reference display and metadata demo	examples/platform/esp/esp32p4
Raspberry Pi Pico	C++ SPI receive and MicroPython SDK module examples	examples/platform/rpi/pico
Raspberry Pi Pico 2	Wiring, serial metadata stream, inference FPS benchmark, and production test examples	examples/platform/rpi/pico2
Raspberry Pi Pico W	Person-detection ROI MVP and SPI receive test examples	examples/platform/rpi/pico_w
Raspberry Pi 5	MIPI preview, Linux I2C/SPI metadata test, and I2C payload model operations	examples/platform/rpi/rpi5
Nordic nRF52840 DK	MicroPython SDK module SPI metadata smoke test	examples/platform/nordic/nrf52840_dk/micropython_imx500_spi_receive
Other MCU platforms	Port by implementing the SDK adapter callbacks	Integration guide: 中文, English, 日本語

Quick Start: ESP32-P4

Use the ESP32-P4 example if you want the shortest path to a visible demo. It combines:

• MIPI CSI camera video input
• I2C control for the IMX500 module
• SPI metadata and tensor payload access
• MIPI DSI LCD output

git clone --recursive https://github.com/ArduCAM/imx500-mcu-sdk.git
cd imx500-mcu-sdk
git submodule update --init --recursive

Then follow the board wiring and project-specific notes in examples/platform/esp/esp32p4/README.md.

Port To A New MCU

Most ports follow the same sequence:

1. Add imx500_mcu_sdk.cmake to your CMake project.
2. Add ${IMX500_MCU_SDK_SRC_FILES} to your target.
3. Apply imx500_mcu_sdk_apply_config(your_target).
4. Implement platform I2C, SPI, delay, and optional log callbacks.
5. Register callbacks with register_i2c_driver(...), register_spi_driver(...), and optionally register_printf(...).
6. Call imx500_open(...), stream_on(), read_metadata(...), and parse_metadata(...).
7. Convert parsed tensors into product events with your post-processing logic.

For the detailed integration manual, see 中文, English, or 日本語.

Metadata To Product Events

The IMX500 module produces sensor-side AI output. Your MCU firmware typically turns that raw output into application logic:

SPI metadata frame
    -> parsed IMX500 metadata
    -> output tensor descriptors
    -> host-side post-processing
    -> product event

Example events include person_count = 3, zone_occupied = true, package_detected = true, or gesture = wake.

Repository Layout

Path	Purpose
ArducamIMX500SDK.h/.cc	Public SDK API and core implementation
ai_driver.h/.c	Platform callback registration interfaces
imx500_mcu_sdk.cmake	SDK source collection and compile-time configuration
python_bindings/	Optional Python host tools and MicroPython user module support
docs/	API landing page, performance benchmark, and integration documentation
examples/	Platform reference projects
third_party/flatbuffers/	FlatBuffers dependency used by network-info parsing

Troubleshooting

• If FlatBuffers headers are missing, run git submodule update --init --recursive.
• If the module is not detected, check power, ground, I2C wiring, and the platform adapter callbacks.
• If metadata reads fail, check SPI pin direction carefully. Camera-side SPI_TX connects to MCU-side RX; camera-side SPI_RX connects to MCU-side TX.
• If the video stream works but AI output does not, confirm the selected metadata format, firmware/network-info loading path, and SPI data-forwarding mode.
• If you changed the MIPI output resolution, rebuild with a supported IMX500_MCU_SDK_SENSOR_MIPI_RESOLUTION value.

Production Design-In Checklist

Before moving beyond evaluation, confirm the product path, model output, optics, enclosure, firmware flow, factory test, and support expectations.

Need	Go to
Freeze production readiness	Production Design-In Checklist
Review lens, FOV, illumination, enclosure, or mounting	Optical Selection
Build a repeatable station or EOL test	EOL Test
Decide when to contact Arducam	Support Options

Journey Map: From First Signal To Production

First signal
    |
    v
USB validation
    |
    v
Choose a product path
    |-- USB3 UVC deployment
    |-- MIPI / Raspberry Pi / CM5 product
    `-- SPI / MCU low-power product
    |
    v
Model validation
    |
    v
Prototype validation
    |
    v
Production design-in

Path details:

• USB Validation To USB3 UVC Deployment
• MIPI / Raspberry Pi / CM5 Product
• SPI Metadata To MCU Product
• Model Validation To Production
• Train And Deploy A Custom Model

Stage	What you get	Arducam can help with
First signal	Camera detected, SDK opens the module, metadata frame received	Bring-up support and debug workflow
Model validation	Known model or custom model result on IMX500	Model conversion, porting, and post-processing
Prototype	Product event output in your application	Optics, firmware, metadata parsing, and interface review
Production	Stable hardware/software package and test flow	Design-in, factory test, customization, SLA, and long-term supply

License

See the repository license terms before using this SDK in a product.

Need Help?

For module information and production design-in discussions, start from the B0642 IMX500 AI camera module product page or the support options.