Skip to content

Add coreml quant recipes #13265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: gh/abhinaykukkadapu/4/head
Choose a base branch
from
Open

Add coreml quant recipes #13265

wants to merge 7 commits into from
+812 −173

Conversation

abhinaykukkadapu
Copy link
Contributor

@abhinaykukkadapu abhinaykukkadapu commented Aug 10, 2025 •
edited
Loading

Adds coreml quant recipes after FP32/16 recipes added in #13121

Recipes added:

  1. PT2E_INT8_STATIC
  2. PT2E_INT8_WEIGHT_ONLY
  3. INT4_WEIGHT_ONLY_PER_CHANNEL
  4. INT4_WEIGHT_ONLY_PER_GROUP
  5. INT8_WEIGHT_ONLY_PER_CHANNEL
  6. INT8_WEIGHT_ONLY_PER_GROUP
  7. CODEBOOK_WEIGHT_ONLY

@abhinaykukkadapu
Update
ebc30b2 
[ghstack-poisoned]
@abhinaykukkadapu
Copy link
Contributor Author

abhinaykukkadapu commented Aug 10, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 10, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13265

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job, 1 Unrelated Failure

As of commit e33390f with merge base a84b3c9 (image):

NEW FAILURE - The following job has failed:

CANCELLED JOB - The following job was cancelled. Please retry:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 10, 2025
abhinaykukkadapu added a commit that referenced this pull request Aug 10, 2025
@abhinaykukkadapu
Add coreml quant recipes
bf7413d 
ghstack-source-id: 1bb73e0
ghstack-comment-id: 3172341606
Pull-Request: #13265
@abhinaykukkadapu abhinaykukkadapu mentioned this pull request Aug 10, 2025
Open
@abhinaykukkadapu
Update
1207a46 
[ghstack-poisoned]
abhinaykukkadapu added a commit that referenced this pull request Aug 10, 2025
@abhinaykukkadapu
Add coreml quant recipes
801d78a 
ghstack-source-id: 76f6fc8
ghstack-comment-id: 3172341606
Pull-Request: #13265
@abhinaykukkadapu abhinaykukkadapu requested review from kimishpatel and metascroy and removed request for shoumikhin, cccclai and mcr229 August 10, 2025 03:52
abhinaykukkadapu
abhinaykukkadapu commented Aug 10, 2025
View reviewed changes
abhinaykukkadapu added a commit that referenced this pull request Aug 10, 2025
@abhinaykukkadapu
Add coreml quant recipes
c0c32b6 
ghstack-source-id: 76f6fc8
ghstack-comment-id: 3172341606
Pull-Request: #13265
@abhinaykukkadapu
Update
1137ee9 
[ghstack-poisoned]
abhinaykukkadapu added a commit that referenced this pull request Aug 10, 2025
@abhinaykukkadapu
Add coreml quant recipes
7f79b99 
ghstack-source-id: d147f71
ghstack-comment-id: 3172341606
Pull-Request: #13265
@abhinaykukkadapu
Update
0e422bf 
[ghstack-poisoned]
abhinaykukkadapu added a commit that referenced this pull request Aug 11, 2025
@abhinaykukkadapu
Add coreml quant recipes
8882d4a 
ghstack-source-id: cdcd86d
ghstack-comment-id: 3172341606
Pull-Request: #13265
metascroy
metascroy reviewed Aug 11, 2025
View reviewed changes
metascroy
metascroy reviewed Aug 11, 2025
View reviewed changes
metascroy
metascroy reviewed Aug 11, 2025
View reviewed changes
metascroy
metascroy reviewed Aug 11, 2025
View reviewed changes
metascroy
metascroy reviewed Aug 11, 2025
View reviewed changes
metascroy
metascroy reviewed Aug 11, 2025
View reviewed changes
metascroy
metascroy reviewed Aug 11, 2025
View reviewed changes
backends/apple/coreml/recipes/coreml_recipe_provider.py
)
elif recipe_type == CoreMLRecipeType.INT4_WEIGHT_ONLY_PER_GROUP:
group_size = kwargs.pop("group_size", 32)
return self._build_torchao_quantized_recipe(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the compute precision used in these quantization recipes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@metascroy It is FLOAT16, see _get_coreml_lowering_recipe function defaults.

@abhinaykukkadapu
Update
b5fd482 
[ghstack-poisoned]
abhinaykukkadapu added a commit that referenced this pull request Aug 12, 2025
@abhinaykukkadapu
Add coreml quant recipes
7658344 
ghstack-source-id: c72ea76
ghstack-comment-id: 3172341606
Pull-Request: #13265
@abhinaykukkadapu abhinaykukkadapu mentioned this pull request Aug 12, 2025
Open
@abhinaykukkadapu
Update
925cf69 
[ghstack-poisoned]
abhinaykukkadapu added a commit that referenced this pull request Aug 12, 2025
@abhinaykukkadapu
Add coreml quant recipes
c2dc8c1 
ghstack-source-id: 622ea48
ghstack-comment-id: 3172341606
Pull-Request: #13265
@abhinaykukkadapu
Update
e33390f 
[ghstack-poisoned]
abhinaykukkadapu added a commit that referenced this pull request Aug 12, 2025
@abhinaykukkadapu
Add coreml quant recipes
4961772 
ghstack-source-id: 3f0920d
ghstack-comment-id: 3172341606
Pull-Request: #13265
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@metascroy metascroy metascroy left review comments

@digantdesai digantdesai Awaiting requested review from digantdesai

@kimishpatel kimishpatel Awaiting requested review from kimishpatel

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@abhinaykukkadapu @metascroy