[webgpu] Fix the incorrect shapes when profiling#26928
Conversation
fs-eire
left a comment
fs-eire
left a comment
There was a problem hiding this comment.
It looks like the overriden shape is not preserved. Is this OK for the profiling?
I did this intentionally. The overridden shape isn’t very straightforward. Or should we keep both versions? |
I think in this PR you can simply fix the empty issue and continue using the overridden shape. Note: The input shape of each operator can be viewed in the CPU event. However, the program of each operator is not always displayed. For example, GQA is shown, but operators like CopyKVCache, FlashAttentionDecodeQKT, etc., are not shown. 
|
Keeping both versions will result many same shapes if use_override_shape is false, which is also not a good method. Restore to use override shape instead of original shape since this is the actual shape used in shader. |
This pull request refactors how input and output tensor shape
information is stored and accessed in the WebGPU context. Instead of
keeping references to the full input and output tensors, only their
shapes are now stored, which helps avoid issues with accessing released
tensors during profiling.
Before
`"inputs[0] = {1,1,768} inputs[1] = {200064,96,1} inputs[2] = {}
outputs[0] = {} "`
After
`"inputs[0] = {1,1,768} inputs[1] = {200064,96,1} inputs[2] = {19206144}
outputs[0] = {1,1,200064} "`
4 participants
This pull request refactors how input and output tensor shape information is stored and accessed in the WebGPU context. Instead of keeping references to the full input and output tensors, only their shapes are now stored, which helps avoid issues with accessing released tensors during profiling.
Before
"inputs[0] = {1,1,768} inputs[1] = {200064,96,1} inputs[2] = {} outputs[0] = {} "After
"inputs[0] = {1,1,768} inputs[1] = {200064,96,1} inputs[2] = {19206144} outputs[0] = {1,1,200064} "