Overload vec::dequantize to eliminate rounding error for quantized sigmoid

[Xia-Weiwen]

Stack from ghstack (oldest at bottom):

• -> Overload vec::dequantize to eliminate rounding error for quantized sigmoid #114098

Description
Fix #107030
Dequantize X by (x_val - zp) * scale instead of x_val * scale + (-zp * scale) to eliminate rounding error.
Now this overload is used for sigmoid only.

Performance impact:

Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz (Ice Lake)

Test plan
python test_quantization.py TestQuantizedOps.test_sigmoid_dequantize_rounding_error

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/114098

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a7ba41b with merge base 5a96a42 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jgong5]

Maybe we need to revisit other callers and fix them too?

[Xia-Weiwen]

Maybe we need to revisit other callers and fix them too?

Yes. Maybe we can do it one by one if the same issue occurs for other ops so that we can minimize the impact.

[Xia-Weiwen]

Hi @jerryzh168 @salilsdesai @kimishpatel @digantdesai @jianyuh Could you please review this PR? Thanks!

[Xia-Weiwen]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[kimishpatel]

Dequantize X by (x_val - zp) * scale instead of x_val * scale + (-zp * scale) to eliminate rounding error

Why were we doing x_val * scale + (-zp * scale) in the first place? That doesnt sound right.

[Xia-Weiwen]

Dequantize X by (x_val - zp) * scale instead of x_val * scale + (-zp * scale) to eliminate rounding error

Why were we doing x_val * scale + (-zp * scale) in the first place? That doesnt sound right.

I guess it was about making use of fma to speed up the computation.

[jgong5]

I guess it was about making use of fma to speed up the computation.

It is bound by mem access, not sure if fma really matters here.