Zephyr: Add MobileNetV2 image classification sample with Ethos-U NPU#19131
Zephyr: Add MobileNetV2 image classification sample with Ethos-U NPU#19131psiddh merged 19 commits intopytorch:mainfrom
Conversation
Add a new Zephyr sample that runs a quantized INT8 MobileNetV2 model on Arm Ethos-U NPU using ExecuTorch. The sample classifies a static 224x224x3 RGB test image into 1000 ImageNet classes and prints the top-5 predictions. Validated end-to-end on Alif Ensemble E8 DevKit (Cortex-M55 + Ethos-U55 256 MAC) achieving 19ms inference with 100% NPU delegation (110 ops). This addresses part of pytorch#17654 (Zephyr: Expand samples and documentation) by adding a second sample app (MV2) beyond the existing hello-executorch. Authored with assistance from Claude.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19131
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ⏳ No Failures, 276 PendingAs of commit e424fba with merge base 9207001 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
This PR needs a
|
There was a problem hiding this comment.
Pull request overview
Adds a new Zephyr sample application that runs a quantized MobileNetV2 image classification model with Arm Ethos‑U delegation via ExecuTorch, including build/test metadata and board-specific configuration.
Changes:
- Introduce
mv2-ethosuZephyr sample that loads an embedded.pteand runs inference on a static 224×224×3 RGB input, printing top‑5 classes. - Add Zephyr build configuration (Kconfig/prj.conf/CMake) to embed the model, size allocators, and selectively build portable ops when needed.
- Add board configs/overlay for Corstone-300/320 FVP to enable Ethos‑U and place SRAM appropriately for DMA access.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| zephyr/samples/mv2-ethosu/src/main.cpp | Sample runtime: load PTE, allocate arenas, set inputs, execute, print top‑5. |
| zephyr/samples/mv2-ethosu/sample.yaml | Zephyr sample metadata and build-only test entries for Corstone FVPs. |
| zephyr/samples/mv2-ethosu/prj.conf | Enables ExecuTorch/C++17 and increases stack/heap + allocator pool sizes. |
| zephyr/samples/mv2-ethosu/boards/mps4_corstone320_fvp.conf | Enables Zephyr Ethos‑U driver for Corstone-320 FVP. |
| zephyr/samples/mv2-ethosu/boards/mps3_corstone300_fvp.conf | Enables Zephyr Ethos‑U driver for Corstone-300 FVP. |
| zephyr/samples/mv2-ethosu/boards/mps3_corstone300_fvp.overlay | Routes Zephyr SRAM to ISRAM so Ethos‑U DMA can access scratch buffers. |
| zephyr/samples/mv2-ethosu/README.md | Usage docs: model export command, build commands, and expected output. |
| zephyr/samples/mv2-ethosu/Kconfig | Sample-specific Kconfig for allocator pool sizes (MV2 defaults). |
| zephyr/samples/mv2-ethosu/CMakeLists.txt | Embeds PTE into header, optional selective portable ops build, links delegate. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Remove deprecated executorch_delegate_EthosUBackend_registered() call; linking executorch_delegate_ethos_u auto-registers the backend - Add model_pte_runtime ISRAM copy for Corstone FVP DMA accessibility - Fix ET_LOG formatting to pass linter (one arg per line) - Add explicit ScalarType::Byte case and error for unsupported types in print_top_k instead of silent uint8 fallback - Add comment explaining ISRAM copy vs direct MRAM access tradeoff
- Remove unused runner_util/inputs.h include from main.cpp - Add BSD license header to mv2_input.h for repo consistency
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Use Error log level for all failure paths instead of Info - Replace ET_CHECK with graceful error log + return for get_outputs - Use Python3_EXECUTABLE instead of hardcoded python in CMake - Add RESULT_VARIABLE/ERROR_VARIABLE checks for gen_oplist.py - Fix grammar in CMake status message - Apply lint formatting suggestions
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Remove BufferCleanup UB: replace with plain Error return since buffers are arena-allocated (not malloc), so free() would be undefined behavior - Add Corstone-300 board-specific pool size overrides (768KB each) to fit within 2 MiB ISRAM budget - Fix README module path to use in-repo invocation - Apply lintrunner formatting to mv2_input.h
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add <cinttypes> for PRIx32 portability - Check memory_planned_buffer_size Result before .get() - Validate single tensor input (reject multi-input models) - Remove Twister test entries from sample.yaml since CMake hard-fails without ET_PTE_FILE_PATH which Twister cannot provide
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Use exact NCHW tensor shape [1, 3, 224, 224] in README and log output - Clarify board-specific .conf files vs DTS overlay files in README - Replace Zephyr __aligned() macro with __attribute__((aligned(4))) in mv2_input.h for portability without Zephyr headers
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add compile-time guard ET_ARM_MODEL_PTE_DMA_ACCESSIBLE to skip the SRAM copy of model_pte on boards where MRAM is DMA-accessible, avoiding RAM exhaustion for large models - Log actual output tensor class count instead of hardcoded 1000
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } else if (input_size == tensor_meta->nbytes()) { | ||
| ET_LOG( | ||
| Info, | ||
| "Copying input data to tensor (%lu bytes)", | ||
| static_cast<unsigned long>(input_size)); | ||
| std::memcpy(data_ptr, input_data, input_size); |
There was a problem hiding this comment.
If the model expects an int8 input tensor (ScalarType::Char), this branch memcpy’s raw uint8 RGB bytes into an int8 buffer, which reinterprets values 128–255 as negative and corrupts inputs for signed-int8 quantized models (common for Ethos-U). Handle ScalarType::Char explicitly by converting uint8->int8 (e.g., subtract 128 / apply the expected zero-point) instead of memcpy.
…9183) Commit c1731fd added CUDA::curand as a CMake dependency in backends/cuda/CMakeLists.txt for the new GPU-side sampling shim (rand.cu). However, the CI Docker image (install_cuda.sh) did not include libcurand-dev, so the CMake build during pip install fails when EXECUTORCH_BUILD_CUDA=ON is auto-detected. This causes executorch to not be installed at all, breaking the dinov2 export (No module named 'executorch') and parakeet export (No module named 'torchaudio', since torchaudio installation is skipped when the prior pip install step fails) in the cuda-windows CI workflow.
Centralize torch-family version pins in torch_pin.py torch_pin.py is now the single source of truth for torch + the three domain libraries (vision/audio/codec). It exposes a CHANNEL field (nightly/test/release), the four version constants, NIGHTLY_VERSION, and helpers — torch_spec() / torchaudio_spec() / torchcodec_spec() / torchvision_spec() emit the right pip spec, torch_index_url_base() returns the right wheel index, and torch_branch() / torchaudio_branch() / torchvision_branch() derive the upstream release/M.N branch from each package's version. Every consumer — install_requirements.py, the two install_pytorch.sh / utils.sh shell helpers, test_model_e2e.sh, test_wheel_package_qnn.sh, the moshi/mimi install_requirements.sh, the update_pytorch_pin.py script, and the weekly bump workflow — reads through these helpers instead of re-encoding the version strings. Switching to a release candidate is now a one-line change (CHANNEL = "test") plus bumping the four version constants. The header in torch_pin.py walks through the procedure. update_pytorch_pin.py imports CHANNEL / NIGHTLY_VERSION / torch_branch directly (no more regex parsing of the file). For nightly it pins to an immutable SHA looked up by date; for test/release it writes torch_branch() (e.g. "release/2.12") into .ci/docker/ci_commit_pins/pytorch.txt so git checkout follows cherry-picks as they land. The weekly-pytorch-pin-bump workflow is guarded on CHANNEL == "nightly" and uses an in-place re.sub on NIGHTLY_VERSION (the previous `printf '...' > torch_pin.py` would have clobbered the new constants and helpers). test/test_torch_pin.py covers all three channels, all four specs, and the release/M.N branch derivation. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
Wire the mv2-ethosu sample into the existing test-arm-backend-zephyr job in trunk.yml. For each NPU target (ethos-u55, ethos-u85) the CI now exports a quantized MV2, builds the Zephyr sample, and runs it on the matching Corstone FVP. cortex-m55 is skipped since MV2 requires NPU delegation. README restructured with <!-- RUN --> markers so the CI helper run_command_block_from_readme() can extract the build/run commands directly—same pattern used by the hello-executorch sample. zephyr/** added to the PR trigger paths so Zephyr changes get CI coverage. Co-authored-by: Claude <noreply@anthropic.com>
Add a new Zephyr sample that runs a quantized INT8 MobileNetV2 model on Arm Ethos-U NPU using ExecuTorch. The sample classifies a static 224x224x3 RGB test image into 1000 ImageNet classes and prints the top-5 predictions.
Validated end-to-end on Alif Ensemble E8 DevKit (Cortex-M55 + Ethos-U55 256 MAC) achieving 19ms inference with 100% NPU delegation (110 ops).
This addresses part of #17654 (Zephyr: Expand samples and documentation) by adding a second sample app (MV2) beyond the existing hello-executorch.