Skip to content

Conversation

navsud
Copy link
Contributor

@navsud navsud commented Sep 24, 2025

Summary:
reduce_range=True reduces the available bit width by 1, in cases where quant_min, quant_max are not provided. It was originally intended for intel fbgemm kernels but I don't think this quantization setting is relevant for HTP.
Also, PTQ quantization config doesn't use it, so removing it in all the QAT configs. This helped improve the QAT model quality.

Differential Revision: D82867843

Summary:

To save GPU memory `bfloat16` dtype is commonly used for training of LLMs. Currently, the quantizer ignores quantizing the nodes if they are not float32. This change enables quantization of bf16 nodes as well.

Differential Revision: D82866443
Summary:
`reduce_range=True` reduces the available bit width by 1, in cases where quant_min, quant_max are not provided. It was originally intended for intel `fbgemm` kernels but I don't think this quantization setting is relevant for HTP. 
Also, PTQ quantization config doesn't use it, so removing it in all the QAT configs. This helped improve the QAT model quality.

Differential Revision: D82867843
@navsud navsud requested a review from cccclai as a code owner September 24, 2025 22:04
Copy link

pytorch-bot bot commented Sep 24, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14559

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 1 Pending, 5 Unrelated Failures

As of commit 07e6a73 with merge base b3f3111 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 24, 2025
@facebook-github-bot
Copy link
Contributor

@navsud has exported this pull request. If you are a Meta employee, you can view the originating diff in D82867843.

@navsud navsud added the release notes: none Do not include this in the release notes label Sep 24, 2025
@navsud navsud requested a review from shewu-quic September 24, 2025 22:08
@billmguo
Copy link
Contributor

@haowhsu-quic can u help review this PR?

Copy link
Collaborator

@haowhsu-quic haowhsu-quic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.

Copy link
Contributor

@cccclai cccclai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stamp on behalf of qcom's team review

@facebook-github-bot facebook-github-bot merged commit 4622edb into pytorch:main Sep 26, 2025
125 of 151 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported release notes: none Do not include this in the release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants