[WebNN EP] Support Shape op #16282
[WebNN EP] Support Shape op #16282
Conversation
Since WebNN API doesn't support shape op, in the WebNN EP, we calculate the ONNX Shape node output and pass the values to a WebNN's constant as workaround.
onnxruntime/core/providers/webnn/builders/impl/shape_op_builder.cc
Outdated
Show resolved
Hide resolved
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
/azp run ONNX Runtime Web CI Pipeline |
|
Azure Pipelines successfully started running 1 pipeline(s). |
onnxruntime/core/providers/webnn/builders/impl/shape_op_builder.cc
Outdated
Show resolved
Hide resolved
|
/azp run ONNX Runtime Web CI Pipeline |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Thanks Wanming. I'm still thinking ideas on how to mitigate GPU readback cost, because we don't actually want to use Const+Slice followed by a bunch of tiny operators (Gather, Concat...) when the end result is just another operator like Reshape consuming that tiny shape tensor. I think we'll need a heuristic like CUDA has for small tensors to prefer those remain on the CPU EP, along with any immediate successors (until reaching an op like Reshape or Expand which consumes that small tensor and should continue on the GPU again).
Note @guschmue that if we add AddFreeDimensionOverrideByName to the JS session options (I'm really surprised these are missing), then many of those Shape operators will be constant folded out anyway, mitigating this cost. It won't help for Segment Anything because of the dynamic input (that case required static model manipulation), but it will help for Stable Diffusion.
|
/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline |
|
/azp run Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
|
Azure Pipelines successfully started running 9 pipeline(s). |
|
Azure Pipelines successfully started running 6 pipeline(s). |
|
AddFreeDimensionOverrideByName : ok, I'll add it to my todo list |
Since WebNN API doesn't support shape op, in the WebNN EP, we calculate the ONNX Shape node output and pass the values to a WebNN's constant + slice as workaround.