Add Cortex-M e2e integration tests on trunk by psiddh · Pull Request #18311 · pytorch/executorch

psiddh · 2026-03-19T07:13:28Z

Summary

test_cortex_m_e2e.sh: Add proper FVP flags (UART to stdout, telnet
disabled, semihosting stack/heap config) matching runner_utils.py,
use absolute WORK_DIR path, pass dummy -i/-o args to satisfy the
runner's argc>=7 check, capture FVP log and validate BundleIO
PASS/FAIL result, remove self-copy bug.
build_test_runner.sh: Pass --devtools to build_executorch.sh so
libbundled_program.a is built, set relaxed ET_ATOL/ET_RTOL via
--extra_build_flags for int8 quantized model tolerance.
aot_arm_compiler.py: Save .bpte/.pte before ETRecord generation
so a serialization failure doesn't block model export. Wrap
ETRecord in try/except since it hits a known serializer bug with
cortex_m.minimum tensor constants in mv3.

Validated locally: both mv2 and mv3 pass BundleIO verification on
Corstone-300 FVP.

Authored with Claude.

pytorch-bot · 2026-03-19T07:13:32Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18311

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 3 Unrelated Failures

As of commit 18ab2de with merge base 60d57e5 ():

NEW FAILURES - The following jobs have failed:

pull / android / run-emulator (gh)
The process '/usr/local/lib/android/sdk/platform-tools/adb' failed with exit code 224
pull / test-multimodal-linux (gemma3-4b) / linux-job (gh)
RuntimeError: Command docker exec -t 74a99b52991bb7c2c8af3aa0562d8e949db87d8c85076c8e4e07bef836ec0e19 /exec failed with exit code 139
pull / unittest-editable / macos / macos-job (gh)
export/tests/test_target_recipes.py::TestTargetRecipes::test_mv3_model
trunk / unittest-release / macos / macos-job (gh)
examples/models/test/test_export.py::ExportTest::test_mv2_export_to_executorch

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / test-models-macos-cpu (w2l, xnnpack-quantization-delegation) / macos-job (gh)
##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.
pull / unittest-editable / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.
trunk / unittest-release / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-03-19T07:14:14Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Pull request overview

Adds Cortex‑M backend test coverage and introduces a trunk CI end‑to‑end flow that exports selected models as BundleIO .bpte and runs them on Corstone‑300 FVP.

Changes:

Add new Cortex‑M model tests covering common torch op patterns, nn.Module compositions, and torch.nn.functional usage.
Update Cortex‑M test runner build to enable BundleIO support.
Replace the existing trunk Cortex‑M test job with a new matrix e2e job (mv2/mv3) that exports via examples.arm.aot_arm_compiler and runs on FVP.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
backends/cortex_m/test/models/test_torch_functions.py	New composite-model dialect tests for common torch patterns through Cortex‑M pipeline.
backends/cortex_m/test/models/test_nn_modules.py	New dialect tests for popular `nn.Module` blocks (with one xfail).
backends/cortex_m/test/models/test_nn_functional.py	New dialect tests for common `torch.nn.functional` patterns.
backends/cortex_m/test/build_test_runner.sh	Build semihosting runner with `--bundleio` enabled.
.github/workflows/trunk.yml	Introduce `test-cortex-m-e2e` matrix job and switch trunk Cortex‑M coverage to e2e invocation.
.github/workflows/pull.yml	Add a PR-only job that runs Cortex‑M pytest suite.
.ci/scripts/test_cortex_m_e2e.sh	New script exporting `.bpte` and running it on Corstone‑300 FVP.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/workflows/trunk.yml

backends/cortex_m/test/models/test_torch_functions.py

backends/cortex_m/test/models/test_nn_modules.py

backends/cortex_m/test/models/test_nn_functional.py

.ci/scripts/test_cortex_m_e2e.sh

Copilot

Pull request overview

Adds Cortex‑M end-to-end (export → FVP run → BundleIO verification) coverage to trunk CI, and adjusts the Cortex‑M test runner build/export flow to support BundleIO and tolerate known ETRecord generation failures.

Changes:

Add a new trunk CI job (test-cortex-m-e2e) that exports mv2/mv3 for cortex-m55+int8 and runs the resulting .bpte on Corstone‑300 FVP, validating BundleIO PASS/FAIL from logs.
Update Cortex‑M test runner build to enable devtools + BundleIO and relax verification tolerances for quantized models.
Reorder ETRecord generation to occur after saving .pte/.bpte, and make ETRecord generation failures non-fatal.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
`examples/arm/aot_arm_compiler.py`	Moves ETRecord generation after saving artifacts and wraps ETRecord generation in a non-fatal try/except.
`backends/cortex_m/test/build_test_runner.sh`	Builds ExecuTorch with `--devtools`, builds the semihosting runner with `--bundleio`, and sets relaxed atol/rtol via CMake defines.
`.github/workflows/trunk.yml`	Adds a new matrix CI job to run Cortex‑M e2e on FVP for mv2/mv3.
`.ci/scripts/test_cortex_m_e2e.sh`	New e2e script: exports model as `.bpte`, runs FVP with semihosting, and checks logs for BundleIO PASS/FAIL.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

examples/arm/aot_arm_compiler.py

.github/workflows/trunk.yml

.ci/scripts/test_cortex_m_e2e.sh

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.ci/scripts/test_cortex_m_e2e.sh

Fix several issues found during local validation of the e2e CI job: - test_cortex_m_e2e.sh: Add proper FVP flags (UART to stdout, telnet disabled, semihosting stack/heap config) matching runner_utils.py, use absolute WORK_DIR path, pass dummy -i/-o args to satisfy the runner's argc>=7 check, capture FVP log and validate BundleIO PASS/FAIL result, remove self-copy bug. - build_test_runner.sh: Pass --devtools to build_executorch.sh so libbundled_program.a is built, set relaxed ET_ATOL/ET_RTOL via --extra_build_flags for int8 quantized model tolerance. - aot_arm_compiler.py: Save .bpte/.pte before ETRecord generation so a serialization failure doesn't block model export. Wrap ETRecord in try/except since it hits a known serializer bug with cortex_m.minimum tensor constants in mv3. Validated locally: both mv2 and mv3 pass BundleIO verification on Corstone-300 FVP. Authored with Claude.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.ci/scripts/test_cortex_m_e2e.sh

- mkdir before realpath so the directory exists on CI - Use a tiny dummy file for -i instead of re-reading the .bpte - Use relative output basename instead of /dev/null - Align fatal error regex with runner_utils.py Authored with Claude.

psiddh · 2026-03-23T03:19:45Z

Newly added jobs/models are passing e2e

rascani · 2026-03-23T15:22:31Z

examples/arm/aot_arm_compiler.py

+        try:
+            generate_etrecord(etrecord_file_name, edge_program_manager_copy, exec_prog)
+            print(f"ETRecord saved as {etrecord_file_name}")
+        except Exception as e:


Can we get a bug filed for cortex-m mv3 etrecord generation throwing an exception?

rascani · 2026-03-23T15:25:12Z

backends/cortex_m/test/build_test_runner.sh

 aten::amax.out"

-${build_executor_runner} --pte=semihosting --target=ethos-u55-128 --output="${build_root_test_dir}" --select_ops_list="${select_ops_list}"
+${build_executor_runner} --pte=semihosting --bundleio --target=ethos-u55-128 --output="${build_root_test_dir}" --select_ops_list="${select_ops_list}" --extra_build_flags="-DET_ATOL=5.0 -DET_RTOL=1.0"


ATOL and RTOL should probably be configurable per-test. Ideally, we also make the defaults as small as possible and override it on tests that need it.

psiddh requested a review from rascani as a code owner March 19, 2026 07:13

Copilot AI review requested due to automatic review settings March 19, 2026 07:13

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 19, 2026

Copilot AI reviewed Mar 19, 2026

View reviewed changes

psiddh force-pushed the cortex_m_e2e branch from 17b1e8c to 7107178 Compare March 19, 2026 07:20

psiddh marked this pull request as draft March 19, 2026 07:21

psiddh mentioned this pull request Mar 19, 2026

Add CI for CortexM Ops (FVP) #17822

Closed

psiddh force-pushed the cortex_m_e2e branch from 7107178 to 2ff1b30 Compare March 20, 2026 19:23

psiddh changed the title ~~Cortex m e2e~~ Add Cortex-M e2e integration tests on trunk Mar 20, 2026

psiddh force-pushed the cortex_m_e2e branch from fe0da90 to fd6e143 Compare March 22, 2026 23:37

psiddh added ciflow/trunk labels Mar 22, 2026

psiddh marked this pull request as ready for review March 22, 2026 23:44

psiddh requested a review from digantdesai as a code owner March 22, 2026 23:44

Copilot AI review requested due to automatic review settings March 22, 2026 23:44

Copilot started reviewing on behalf of psiddh March 22, 2026 23:44 View session

Copilot AI reviewed Mar 22, 2026

View reviewed changes

examples/arm/aot_arm_compiler.py Outdated Show resolved Hide resolved

.github/workflows/trunk.yml Show resolved Hide resolved

.ci/scripts/test_cortex_m_e2e.sh Outdated Show resolved Hide resolved

Copilot AI review requested due to automatic review settings March 22, 2026 23:53

Copilot started reviewing on behalf of psiddh March 22, 2026 23:54 View session

Copilot AI reviewed Mar 22, 2026

View reviewed changes

.ci/scripts/test_cortex_m_e2e.sh Show resolved Hide resolved

psiddh force-pushed the cortex_m_e2e branch from 97ff824 to e9632a2 Compare March 23, 2026 00:01

Update examples/arm/aot_arm_compiler.py

43844a7

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 23, 2026 00:05

psiddh force-pushed the cortex_m_e2e branch from e9632a2 to 43844a7 Compare March 23, 2026 00:05

Copilot started reviewing on behalf of psiddh March 23, 2026 00:05 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

.ci/scripts/test_cortex_m_e2e.sh Show resolved Hide resolved

.ci/scripts/test_cortex_m_e2e.sh Outdated Show resolved Hide resolved

psiddh requested review from AdrianLundell and zingo March 23, 2026 00:54

rascani reviewed Mar 23, 2026

View reviewed changes

This was referenced Mar 23, 2026

ETRecord generation fails for mv3 with cortex_m.minimum tensor constants #18422

Open

ATOL and RTOL should probably be configurable per-test. Ideally, we also make the defaults as small as possible and override it on tests that need it. #18424

Open

rascani approved these changes Mar 24, 2026

View reviewed changes

psiddh merged commit dc084a9 into main Mar 24, 2026
542 of 557 checks passed

psiddh deleted the cortex_m_e2e branch March 24, 2026 19:45

Conversation

psiddh commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

pytorch-bot bot commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18311

❌ 4 New Failures, 1 Cancelled Job, 3 Unrelated Failures

Uh oh!

github-actions bot commented Mar 19, 2026

This PR needs a release notes: label

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

psiddh commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rascani Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

rascani Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

psiddh commented Mar 19, 2026 •

edited

Loading

pytorch-bot bot commented Mar 19, 2026 •

edited

Loading

This PR needs a `release notes:` label

psiddh commented Mar 23, 2026 •

edited

Loading