New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] #2295

Closed

jiyuanzFB wants to merge 1 commit into pytorch:main from jiyuanzFB:export-D52136852

Contributor

jiyuanzFB commented Jan 30, 2024

Summary:

Add FX Kernel benchmark for dynamic quantized gemm step-2
Use quantize_step parameter to differentiate different stages
Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

netlify bot commented Jan 30, 2024 •

edited

Loading

✅ Deploy Preview for pytorch-fbgemm-docs ready!

Name	Link
🔨 Latest commit	`6fdfde4`
🔍 Latest deploy log	https://app.netlify.com/sites/pytorch-fbgemm-docs/deploys/65c2b28a2d8803000867ab82
😎 Deploy Preview	https://deploy-preview-2295--pytorch-fbgemm-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

facebook-github-bot added the cla signed label

Contributor

facebook-github-bot commented Jan 30, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

facebook-github-bot added the fb-exported label

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

d715a1b

…torch#2295)

Summary:

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

ad356b7

…torch#2295)

Summary:

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

jiyuanzFB force-pushed the export-D52136852 branch from 8d8aa59 to ad356b7 Compare

January 31, 2024 04:18

Contributor

facebook-github-bot commented Jan 31, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

1 similar comment

Contributor

facebook-github-bot commented Jan 31, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

fc1f2df

…torch#2295)

Summary:

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

Contributor

facebook-github-bot commented Feb 1, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

a0df5e2

…torch#2295)

Summary:

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

jiyuanzFB force-pushed the export-D52136852 branch from ad356b7 to a0df5e2 Compare

February 1, 2024 00:09

Contributor

facebook-github-bot commented Feb 1, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

2cbe0c9

…torch#2295)

Summary:

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

d24b38e

…torch#2295)

Summary:

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

jiyuanzFB force-pushed the export-D52136852 branch from a0df5e2 to d24b38e Compare

February 1, 2024 00:10

Contributor

facebook-github-bot commented Feb 1, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB force-pushed the export-D52136852 branch from d24b38e to a0df5e2 Compare

February 1, 2024 00:10

Contributor

facebook-github-bot commented Feb 1, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB force-pushed the export-D52136852 branch from a0df5e2 to d24b38e Compare

February 1, 2024 00:11

Contributor

facebook-github-bot commented Feb 1, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

077e23a

…torch#2295)

Summary:
Pull Request resolved: pytorch#2295

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

fbshipit-source-id: 519e842dd60c65fade6c3c37982c6f1628aff4d7

Contributor

facebook-github-bot commented Feb 1, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

a26d5e9

…torch#2295)

Summary:
Pull Request resolved: pytorch#2295

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

fbshipit-source-id: 277559d27bfe1da803918e5e7dffed8b8a8a6c73

jiyuanzFB force-pushed the export-D52136852 branch from d24b38e to a26d5e9 Compare

February 1, 2024 00:17

Contributor

facebook-github-bot commented Feb 1, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

jiyuanzFB pushed a commit to jiyuanzFB/FBGEMM that referenced this pull request


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

e835ceb

…torch#2295)

Summary:
Pull Request resolved: pytorch#2295

- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

Differential Revision: D52136852

fbshipit-source-id: eeabe9bfb94e07d1c336dab18150022d5582dcd6

jiyuanzFB force-pushed the export-D52136852 branch from a26d5e9 to e835ceb Compare

February 1, 2024 00:21

Contributor

facebook-github-bot commented Feb 6, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852


          add dynamic quantize gemm benchmark [step 2: fp16->int8 quantize] (py…

6fdfde4

…torch#2295)

Summary:

- Register the 2nd step operator `qlinear_quant` into FX stack
- Add FX Kernel benchmark for dynamic quantized gemm step-2
- Use `quantize_step` parameter to differentiate different stages
- Separate Net modules for step-2 vs step-1 --

result:
https://fb-my.sharepoint.com/:x:/g/personal/jiyuanz_meta_com/Ec94q-KgmslMtQ7nIYT4240BZUyWiK-iQvP1cBgzfgEDWg?e=DfP82U

1K x 1K: 638 cycles (5.10 us) --> 411 GB/s
2K x 2K: 1200 cycles (9.6 us) --> 873 GB/s

Reviewed By: charliezjw

Differential Revision: D52136852

jiyuanzFB force-pushed the export-D52136852 branch from e835ceb to 6fdfde4 Compare

February 6, 2024 22:28

Contributor

facebook-github-bot commented Feb 6, 2024

This pull request was exported from Phabricator. Differential Revision: D52136852

facebook-github-bot closed this in

44fc10a

facebook-github-bot added the Merged label

Contributor

facebook-github-bot commented Feb 7, 2024

This pull request has been merged in 44fc10a.

PaulZhang12 added a commit to PaulZhang12/FBGEMM that referenced this pull request


          Improve Inference UX, add tests, and add inference API docstrings (py…

46a8c8b

…torch#2295)

Summary: Pull Request resolved: pytorch/torchrec#2295

Differential Revision: D61175593

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment