[ExecuTorch][WebGPU] Add quantize_per_tensor op (int8 buffer path)#21187
[ExecuTorch][WebGPU] Add quantize_per_tensor op (int8 buffer path)#21187JCNTH wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21187
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ You can merge normally! (1 Unrelated Failure)As of commit bef151e with merge base 266e0dc ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
meta-cla
Bot
added
the
CLA Signed
psiddh
left a comment
psiddh
left a comment
There was a problem hiding this comment.
Approving full WebGPU stack
2 participants
Stack from ghstack (oldest at bottom):
Problem: The WebGPU delegate has no per-tensor quantization — the C0 gate for the int8 activation path that the Vulkan delegate's q8ta quantized ops build on. Without it, no int8-input/output op can be served.
Solution: Port
quantized_decomposed.quantize_per_tensor.default(fp32 -> int8), the backend's first non-fp32 storage-buffer op. The int8 output is packed 4 elements per 32-bit word and bound asarray<u32>(the landedq4gswidiom), since WebGPU always allocates buffers and ignores the serializedmemory_layout.Implementation:
QuantizePerTensor.cppregisters only.default(mirrors Vulkan), readsscale/zero_pointat fixed arg indices without = args.back()(robust to the overload's arg count), and guards int8 output /numel % 4 == 0(thearray<u32>binding) / fp32 input / non-null buffers, all fail-loud.quantize_per_tensor.wgslcomputesround(x * inv_scale) + zero_point, clamps to[-128, 127], and packs;inv_scaleis the reciprocal taken in double then cast to f32, bit-matching torch'sround(input * (1.0 / scale)).WebGPUTensorgains anis_int8flag (distinguishes int8 from uint8/bool, which share a 1-byte size) so the handler can reject a non-int8 output. Mirrors Vulkanruntime/graph/ops/glsl/q8ta_quantize.glsl+impl/QuantizeDequantize.cpp.Differential Revision: D112257614