[Vulkan] Enable copying QInt8 and QInt32 tensors from cpu to vulkan. #90357
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Copying QInt8 and QInt32 from cpu to vulkan: - Added shader nchw_to_image_int8 - Added shader nchw_to_image_int32 Copying QInt8 and QInt32 from vulkan to cpu Note: This functionality is currently disabled until issues on Android are resolved. - Added shader image_to_nchw_int32 - QInt8 works with the same existing image_to_nchw_quantized shaders Added multiple tests for each supported dtype: - cpu_to_vulkan_and_dequantize: These tests check the correctness of copying quantized cpu tensor to vulkan by comparing the output of the following: - cpu float tensor -> quantize -> to vulkan -> dequantize -> to cpu - cpu float tensor -> quantize -> dequantize - cpu_to_vulkan_and_vulkan_to_cpu (currently disabled until copying vulkan quantized to cpu is enabled): These tests check the correctness of copying from cpu to vulkan and from vulkan to cpu by creating a random cpu float tensor, quantizing it, then copying it to vulkan, then back to cpu and comparing the output tensor to the original quantized tensor. - quantize_per_tensor_and_vulkan_to_cpu (currently disabled until copying vulkan quantized to cpu is enabled): These tests check the correctness of copying quantized tensor from vulkan to cpu by comparing the output of the following: - cpu float tensor -> to vulkan -> quantize -> to cpu - cpu float tensor -> quantize Test Plan: On Mac ``` cd ~/fbsource buck1 run -c pt.vulkan_full_precision=1 //xplat/caffe2:pt_vulkan_quantized_api_test_binAppleMac\#macosx-arm64 ``` On Android ``` cd ~/fbsource buck1 build -c ndk.custom_libcxx=false -c pt.enable_qpl=0 -c pt.vulkan_full_precision=1 //xplat/caffe2:pt_vulkan_quantized_api_test_binAndroid\#android-arm64 --show-output adb push buck-out/gen/xplat/caffe2/pt_vulkan_quantized_api_test_binAndroid\#android-arm64 /data/local/tmp/vulkan_quantized_api_test adb shell "/data/local/tmp/vulkan_quantized_api_test" ``` Reviewed By: kimishpatel Differential Revision: D41654287 fbshipit-source-id: 649d5a6b966242c9c8993ea1b7ec848fc4c61d85
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90357
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 Failures, 1 PendingAs of commit b3d1417: The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
This pull request was exported from Phabricator. Differential Revision: D41654287 |
SS-JIA
approved these changes
Dec 7, 2022
|
@pytorchbot merge (Initiating merge automatically since Phabricator Diff has merged) |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
kulinseth
pushed a commit
to kulinseth/pytorch
that referenced
this pull request
Dec 10, 2022
…ytorch#90357) Summary: Copying QInt8 and QInt32 from cpu to vulkan: - Added shader nchw_to_image_int8 - Added shader nchw_to_image_int32 Copying QInt8 and QInt32 from vulkan to cpu Note: This functionality is currently disabled until issues on Android are resolved. - Added shader image_to_nchw_int32 - QInt8 works with the same existing image_to_nchw_quantized shaders Added multiple tests for each supported dtype: - cpu_to_vulkan_and_dequantize: These tests check the correctness of copying quantized cpu tensor to vulkan by comparing the output of the following: - cpu float tensor -> quantize -> to vulkan -> dequantize -> to cpu - cpu float tensor -> quantize -> dequantize - cpu_to_vulkan_and_vulkan_to_cpu (currently disabled until copying vulkan quantized to cpu is enabled): These tests check the correctness of copying from cpu to vulkan and from vulkan to cpu by creating a random cpu float tensor, quantizing it, then copying it to vulkan, then back to cpu and comparing the output tensor to the original quantized tensor. - quantize_per_tensor_and_vulkan_to_cpu (currently disabled until copying vulkan quantized to cpu is enabled): These tests check the correctness of copying quantized tensor from vulkan to cpu by comparing the output of the following: - cpu float tensor -> to vulkan -> quantize -> to cpu - cpu float tensor -> quantize Test Plan: On Mac ``` cd ~/fbsource buck1 run -c pt.vulkan_full_precision=1 //xplat/caffe2:pt_vulkan_quantized_api_test_binAppleMac\#macosx-arm64 ``` On Android ``` cd ~/fbsource buck1 build -c ndk.custom_libcxx=false -c pt.enable_qpl=0 -c pt.vulkan_full_precision=1 //xplat/caffe2:pt_vulkan_quantized_api_test_binAndroid\#android-arm64 --show-output adb push buck-out/gen/xplat/caffe2/pt_vulkan_quantized_api_test_binAndroid\#android-arm64 /data/local/tmp/vulkan_quantized_api_test adb shell "/data/local/tmp/vulkan_quantized_api_test" ``` Reviewed By: kimishpatel Differential Revision: D41654287 Pull Request resolved: pytorch#90357 Approved by: https://github.com/SS-JIA
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Copying QInt8 and QInt32 from cpu to vulkan:
Copying QInt8 and QInt32 from vulkan to cpu
Note: This functionality is currently disabled until issues on Android are resolved.
Added multiple tests for each supported dtype:
These tests check the correctness of copying quantized cpu tensor to vulkan by comparing the output of the following:
(currently disabled until copying vulkan quantized to cpu is enabled):
These tests check the correctness of copying from cpu to vulkan and from vulkan to cpu by creating a random cpu float tensor, quantizing it, then copying it to vulkan, then back to cpu and comparing the output tensor to the original quantized tensor.
(currently disabled until copying vulkan quantized to cpu is enabled):
These tests check the correctness of copying quantized tensor from vulkan to cpu by comparing the output of the following:
Test Plan:
On Mac
On Android
Reviewed By: kimishpatel
Differential Revision: D41654287