[ExecuTorch][WebGPU] Route exact prefill shapes to the BK64 GEMM#21130
[ExecuTorch][WebGPU] Route exact prefill shapes to the BK64 GEMM#21130JCNTH wants to merge 4 commits into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21130
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 1 Cancelled JobAs of commit 385c00c with merge base 86c3470 ( CANCELLED JOB - The following job was cancelled. Please retry:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
SS-JIA
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
Stack from ghstack (oldest at bottom):
Llama prefill drives the ordinary quantized-linear projections at a small set
of fixed batch-row counts, and the generic Steel schedule leaves throughput on
the table for them. This adds a frozen BK64 Steel quantized-GEMM route for the
exact accepted Llama ordinary-projection shapes at live M128, M508, and M512,
selected only when the capability and dynamic-route guards all pass. M511 and
other prefill sizes stay on the generic Steel schedule and M1 stays on bicol
decode, so the route fails closed outside its accepted shapes. Mirrors Vulkan
xplat/executorch/backends/vulkan/runtime/graph/ops/glsl/q4gsw_linear_gemm__w_4x8.glsl
(the 4-bit grouped-symmetric-weight GEMM), specialized here for the BK64 tile.
Key changes:
BK64-tiled q4gsw prefill GEMM kernel.
and dynamic re-entry into and out of the BK64 route.
@exported-using-ghexport
Differential Revision: D113171739
Differential Revision: D113171739