Skip to content

Conversation

@zingo
Copy link
Collaborator

@zingo zingo commented Feb 13, 2025

Create model test script and break up run.sh into smaller scripts that can be used from both places.

New model test script can be used like this:

$ python3 backends/arm/test/test_model.py --build_libs
$ python3 backends/arm/test/test_model.py --target=ethos-u55-64 --model=lstm
$ python3 backends/arm/test/test_model.py --target=ethos-u85-256 --model=mul

This change also makes it possible to run the things in run.sh in separate steps like this:

  1. Build needed libs

$ backends/arm/scripts/build_executorch.sh
$ backends/arm/scripts/build_portable_kernels.sh --portable_kernels=OPS
$ backends/arm/scripts/build_quantized_ops_aot_lib.sh

  1. Build

$ python3 -m examples.arm.aot_arm_compiler --target=ethos-u85-128 --delegate --quantize --so_library=PATH/libquantized_ops_aot_lib.so --model_name=MODEL

  1. Build target executable ELF_FILE

$ backends/arm/scripts/build_executorch_runner.sh --pte=PTE_FILE --target=ethos-u85-128

  1. Test target executable with FVP

$ backends/arm/scripts/run_fvp.sh --elf=ELF_FILE --target=ethos-u85-128

cc @digantdesai @freddan80 @per @oscarandersson8218

@pytorch-bot
Copy link

pytorch-bot bot commented Feb 13, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8460

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Pending, 1 Unrelated Failure

As of commit 34738e9 with merge base f0ef51c (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 13, 2025
@zingo zingo added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: arm Changes to the ARM backend delegate ciflow/trunk labels Feb 13, 2025
@zingo zingo requested a review from digantdesai February 13, 2025 14:37
@zingo
Copy link
Collaborator Author

zingo commented Feb 13, 2025

@digantdesai there is 2 files outside arm folders are they OK to merge? They are still kind of Arm files, related to arm github testing and the arm delegate guide. 😊

If/when this goes in there is a second patch waiting to be uploaded that add BundleIO testing to this flow so we get input/output verification on our FVP model tests 😊

Edit: Got feedback in other channels by Digant that it was OK to merge this.

@zingo zingo changed the title Arm backend: Create script for model testing ans split run.sh example script Arm: Create script for model testing and split run.sh example script Feb 17, 2025
Copy link
Collaborator

@oscarandersson8218 oscarandersson8218 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. CI failure should be unrelated.

@zingo zingo force-pushed the Arm-backend-Create-script-for-model-testing branch from 3589d57 to f2cb44a Compare February 19, 2025 10:28
Create model test script and break up run.sh into smaller scripts that
can be used from both places.

This makes it possible to run things in run.sh in separate steps
like this:

1. Build needed libs
backends/arm/scripts/build_executorch.sh
backends/arm/scripts/build_portable_kernels.sh --portable_kernels=<OPS>
backends/arm/scripts/build_quantized_ops_aot_lib.sh

2. Build <PTE FILE>
python3 -m examples.arm.aot_arm_compiler --target=ethos-u85-128 --delegate --quantize --so_library=<PATH>/libquantized_ops_aot_lib.so --model_name=<MODEL>

3. Build target executable <ELF FILE>
backends/arm/scripts/build_executorch_runner.sh --pte=<PTE_FILE> --target=ethos-u55-128

4. Test target executable in FVP
backends/arm/scripts/run_fvp.sh --elf=<ELF_FILE> --target=ethos-u85-128

Signed-off-by: Zingo Andersen <zingo.andersen@arm.com>
Change-Id: I0aefad9e727bc7265e045356d0440d35583957d7
@zingo zingo force-pushed the Arm-backend-Create-script-for-model-testing branch from f2cb44a to 34738e9 Compare February 19, 2025 11:12
@zingo
Copy link
Collaborator Author

zingo commented Feb 19, 2025

Fails are unrealted

@zingo zingo merged commit fea3684 into pytorch:main Feb 19, 2025
106 of 109 checks passed
@zingo zingo deleted the Arm-backend-Create-script-for-model-testing branch February 26, 2025 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: arm Changes to the ARM backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants