Arm backend: Consolidate CI testing#19696
Conversation
Testing is done via test_arm_backed.sh instead of different scripts. OOBT tests in CI is part of the test matrix instead of separate job descriptions doing the same thing. Github job names all start with test-arm-backend making it easier know what jobs belongs to and to filter in the HUD. Signed-off-by: Zingo Andersen <Zingo.Andersen@arm.com> Change-Id: Ic36bc41eb1a0581c68aa5bc3c15cdf4377ed981d
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19696
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 1 New FailureAs of commit 76a4602 with merge base b4a9e72 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
There was a problem hiding this comment.
Pull request overview
This PR consolidates Arm backend CI coverage by routing out-of-the-box (OOTB) notebook/e2e checks through backends/arm/test/test_arm_backend.sh, simplifying the workflow matrix and standardizing job naming.
Changes:
- Add OOTB test entrypoints to
test_arm_backend.shthat delegate totest_arm_ootb.sh. - Add a dedicated requirements file for OOTB notebook execution (
notebook,nbconvert) and install it on-demand in the OOTB script. - Update CI workflows to include OOTB tests in the Arm backend test matrices and remove the standalone OOTB CI job; rename the PR workflow Arm job.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| backends/arm/test/test_arm_ootb.sh | Adds repo-root/requirements discovery and on-demand install for notebook execution deps. |
| backends/arm/test/test_arm_backend.sh | Introduces new test_* entrypoints to run OOTB suites via test_arm_ootb.sh. |
| backends/arm/requirements-arm-ootb-test.txt | New requirements list for OOTB notebook conversion/execution. |
| backends/arm/README.md | Documents the newly added OOTB test entrypoints. |
| .github/workflows/trunk.yml | Moves OOTB coverage into the existing Arm backend test matrices; removes standalone OOTB job. |
| .github/workflows/pull.yml | Renames the Arm backend “no driver” job (but currently introduces a YAML syntax issue). |
Comments suppressed due to low confidence (1)
backends/arm/test/test_arm_ootb.sh:40
install_ootb_test_requirementsfalls back topip install ...if the imports fail. Using barepipcan install into a different interpreter thanpython3(especially in conda), which can still leavepython3/jupytermissing the modules. Preferpython3 -m pip install -r "${ootb_requirements}"(and ideally check thatjupyter nbconvertis available after install).
install_ootb_test_requirements() {
python3 - <<'PY' || pip install -r "${ootb_requirements}"
import notebook # noqa: F401
import nbconvert # noqa: F401
PY
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
.github/workflows/trunk.yml:362
- PR description mentions running tests via
test_arm_backed.sh, but the script used throughout the repo/workflows istest_arm_backend.sh. Consider correcting the PR description to avoid confusion for reviewers and future readers.
bash .ci/scripts/setup-conda.sh
eval "$(conda shell.bash hook)"
Testing is done via test_arm_backed.sh instead of different scripts. OOBT tests in CI is part of the test matrix instead of separate job descriptions doing the same thing.
GitHub job names all start with test-arm-backend making it easier know what jobs belongs to and to filter in the HUD.
Test plan
Github testing
cc @digantdesai @freddan80 @per @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani