Skip to content

Add support for MobileNet v2 in Ethos-U55 and arm baremetal runner #3056

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 11 commits into from
Closed

Add support for MobileNet v2 in Ethos-U55 and arm baremetal runner #3056

wants to merge 11 commits into from

Conversation

@robell
Copy link
Collaborator

@robell robell commented Apr 16, 2024

  • Builds on TOSA support for mv2
  • Currently has accuracy issues for on-target execution.
  • Adding support mv2 from examples/models (torchvision mv2)
  • Adds ability to use portable Q/DQ ops to enable float signature quantized models (to be improved with int signature when possible to enable integer images rather than int -> float -> quant flow.
  • Adds permute for ethos-u backend to enable NCHW to NHWC (which we'll aim to replace with a compiler pass on entry/exit of the subgraph, or the NHWC support when available.
  • small fix to enable qantized ops in runtime for the IOQDQ pass, and build/register those kernels/ops in bare metal build

@pytorch-bot
Copy link

pytorch-bot bot commented Apr 16, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/3056

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 6f4ce19 with merge base cc12d9b (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 16, 2024
@robell robell requested a review from digantdesai April 16, 2024 11:54
@robell robell self-assigned this Apr 16, 2024
@robell robell added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk labels Apr 16, 2024
@robell robell changed the title Add support of mv2 in arm baremetal runner Add support for MobileNet v2 in Ethos-U55 and arm baremetal runner Apr 16, 2024
digantdesai
digantdesai approved these changes Apr 16, 2024
View reviewed changes
Copy link
Contributor

@digantdesai digantdesai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. Thanks @robell.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Apr 18, 2024

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

2 similar comments
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Apr 19, 2024

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Apr 23, 2024

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@robell robell force-pushed the mv2_rebase2 branch 3 times, most recently from f346e0f to c2442c4 Compare April 23, 2024 17:33
@digantdesai
Copy link
Contributor

digantdesai commented Apr 24, 2024

Sigh.. can you rebase please?

@robell robell force-pushed the mv2_rebase2 branch 5 times, most recently from 5996cbf to 0790d93 Compare April 26, 2024 10:23
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Apr 26, 2024

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

1 similar comment
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 1, 2024

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

robell added 8 commits May 2, 2024 08:51
@robell
Adding mn2 invocation to run.sh and use correct model section
77b5666 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
@robell
Add flag to register custom ops (like q/dq nodes) in aot_arm_compiler
c2c3544 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
@robell
Add pass to maintain IO quantization
1f28d1d 
Currently only enabled in the aot_ compile flow as this requires
AoT kernel registration.

Signed-off-by: Rob Elliott <robert.elliott@arm.com>
Change-Id: I0c41b3a34613c900fc3f350f4f83b41378ac0add
@robell
Enable mv2 quantized to run on Corstone-300 * NOTE: this has accuracy…
4deed60 
… issues until further fixes * enable further input types and layouts * add optional temp support for reordering NCHW * include portable quantized ops for IOQDQ fallback

Optional reordering  will be removed once NHWC support is introduced.

Signed-off-by: Rob Elliott <robert.elliott@arm.com>
Change-Id: Iba2d4954fa14bc937584e59cb2ce82f4cdc1205d
@robell
Fix for ambiguous operator overload.
d100313 
The templated function with this change has an ambigious overload, making it unambiguous by casting up to the templated type in this quantized function
@robell
lintfix
3d848fe 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
@robell
documented permute and quantize_io compilespec properties
9b1a964 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
@robell
popping arm_partition metadata after use
fb4befe 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
robell added 2 commits May 2, 2024 08:54
@robell
Refactor ArmBackend permute for readability
ebfd6ea 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
@robell
Add FVP option to increase ITCM
467c3a5 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
@robell
Moving aot lib build to cmake
6f4ce19 
Signed-off-by: Rob Elliott <robert.elliott@arm.com>
@robell
Copy link
Collaborator Author

robell commented May 2, 2024

@digantdesai or @mergennachin could we import this now, thanks to the fix in #3449 this now passes when rebased.

@robell robell mentioned this pull request May 2, 2024
Closed
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 2, 2024

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 3, 2024

@digantdesai merged this pull request in 623f0e0.

@oscarandersson8218 oscarandersson8218 mentioned this pull request Jul 3, 2024
Closed
@freddan80 freddan80 mentioned this pull request Aug 15, 2024
Closed
@robell robell deleted the mv2_rebase2 branch October 1, 2024 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@digantdesai digantdesai digantdesai approved these changes

Assignees

@robell robell

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@robell @facebook-github-bot @digantdesai