[ET-VK] Quantized Int8 Convolution + Linear

[SS-JIA]

Stack from ghstack (oldest at bottom):

• [ET-VK][AOT] Enable exporting Q8 Quantized Linear + Convolution #13812
• -> [ET-VK] Quantized Int8 Convolution + Linear #13811
• [ET-VK] Introduce custom op correctness + speed testing suite & add vulkan operator testing to CI #13810
• [ET-VK][ez] Add ability to check for dot product extension support + upgrade glslc #13809

Title says it all!

This PR adds implementations for int8 quantized convolution and linear layers. Convolution is implemented as matrix multiplication under the hood by using the im2col procedure.

For both linear and convolution, two versions are implemented:

1. q8ta_q8csw variant which quantized the input tensor and then performs integer accumulation via the int8 dot product extension
2. q8csw variant which dequantized the weight tensor in-shader and performs floating point accumulation.

The second one is needed to provide an alternative path for executing quantized models if the target GPU does not support int8 dot product extension.

These new ops are tested via the custom op testing + benchmarking framework introduced in the previous diff.

Differential Revision: D81323424

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13811

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• ROCm CI/CD workflows failing due to : download from https://api.github.com/repos/pytorch/pytorch timed out.

❌ 9 New Failures

As of commit 501f89c with merge base e2098f8 ():

NEW FAILURES - The following jobs have failed:

• pull / android / run-emulator (gh)
  The process '/usr/bin/sh' failed with exit code 255
• pull / test-vulkan-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 7f9a4cf5eaf3238767ccd1703ecc257ac8fbded5d9b4ebe478a68217409ed9e7 /exec failed with exit code 1
• pull / test-vulkan-operators-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 7b435e6e4ac257c2caf91c4badf05bd30095d28818b31021a43f690076d67a80 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t b4c66360adbbf1e989d1752725d040354738678cd12002ab52c7f91fc8fc750c /exec failed with exit code 3
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t f96a82f00e2d8103b23695b1e938c0fa00549fc34e02c9e5cad8fa5463dca571 /exec failed with exit code 3
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 6edb945053a3bf26a71c12fa0b15cd30c0320e8ac7907c696298aeec4b1b2cbb /exec failed with exit code 3
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 6459046a092e19c403e49b24c4166bc8e37fc2c0dc5a4123c09f50e2783a5e22 /exec failed with exit code 3
• pull / unittest-editable / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D81323424