[JS/WebGPU] Squeeze operator implementation by visheratin · Pull Request #16024 · microsoft/onnxruntime

visheratin · 2023-05-20T18:51:53Z

Description

This PR adds an implementation of the Squeeze operator to WebGPU JSEP. The implementation follows the operator schema and allows one or two inputs.

How was it tested

I created two models. Without axes:

import onnx.helper

node = onnx.helper.make_node(
    "Squeeze",
    inputs=["T"],
    outputs=["y"],
)
graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 1, 4, 5])], 
    [onnx.helper.make_tensor_value_info("y", 1, [3, 4, 5])])
onnx.save(onnx.helper.make_model(graph), "squeeze.onnx")

And with axes:

import onnx.helper

node = onnx.helper.make_node(
    "Squeeze",
    inputs=["T", "axes"],
    outputs=["y"],
)
graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 1, 4, 5]), onnx.helper.make_tensor_value_info("axes", 7, [1])], [onnx.helper.make_tensor_value_info("y", 1, [3, 4, 5])])
onnx.save(onnx.helper.make_model(graph), "squeeze-dim.onnx")

I compiled the runtime using @fs-eire's instructions.
I ran the test models in the browser using this minimal setup:

<html>
    <script src=".\dist\ort.webgpu.min.js"></script>
    <script>
        async function run() {
            const session = await ort.InferenceSession.create('squeeze-dim.onnx', {executionProviders: ['webgpu']});
            console.log(session);
            const input = new ort.Tensor('float32', new Float32Array(60), [3, 1, 4, 5]);
            const dim = new ort.Tensor('int64', [-3n], [1]);
            const output = await session.run({ "T": input, "axes": dim });
            console.log(output);
        }
        run();
    </script>
</html>

Motivation and Context

Improve operator coverage for WebGPU JSEP.

visheratin · 2023-05-20T18:53:03Z

@microsoft-github-policy-service agree

satyajandhyala

Please check ONNX Squeeze documentation here
The Sequeeze versions are 1, 11 and 13. So we group, from 1 to 10, from 11 to 12 and from 13 and up.

guschmue · 2023-05-25T17:18:35Z

/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,ONNX Runtime Web CI Pipeline,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline

azure-pipelines · 2023-05-25T17:19:14Z

Azure Pipelines successfully started running 10 pipeline(s).

guschmue · 2023-05-25T17:50:09Z

/azp run Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed

azure-pipelines · 2023-05-25T17:50:36Z

Azure Pipelines successfully started running 6 pipeline(s).

visheratin · 2023-05-25T23:45:44Z

I see that some pipelines failed, but I don't know if I can fix it.

fs-eire · 2023-05-26T00:12:07Z

as suggested in the error message, you may need to run "npm run build:doc" in folder /js/web and checkin the changes of updated file under /js/docs/

visheratin · 2023-05-26T00:28:23Z

I tried it, but the command didn't generate any committable changes. Maybe I didn't add something that can be used by generate-operator-md.ts?

fs-eire · 2023-05-26T01:20:55Z

if you have file generate-operator-md.ts in your source code, you need to merge latest main branch, where we added generate-webgpu-operator-md.ts and js/web/docs/webgpu-operators.md

please make sure do npm ci in /js/web before running npm run build:doc.

visheratin · 2023-05-26T01:30:01Z

Thanks! Now the changes were generated.

guschmue · 2023-05-26T16:26:28Z

/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,ONNX Runtime Web CI Pipeline,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline

guschmue · 2023-05-26T16:26:57Z

/azp run Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed

azure-pipelines · 2023-05-26T16:27:08Z

Azure Pipelines successfully started running 10 pipeline(s).

azure-pipelines · 2023-05-26T16:27:20Z

Azure Pipelines successfully started running 6 pipeline(s).

guschmue · 2023-05-26T22:55:57Z

@visheratin , thanks for the contribution!
We sure apricate help with adding ops for webgpu.

hariharans29 · 2023-05-27T00:41:45Z

Should this be uncommented now -

onnxruntime/js/web/test/suite-test-list.jsonc

Line 1229 in 415c26e

// "test_squeeze_negative_axes",

?

fs-eire · 2023-05-27T01:27:47Z

Should this be uncommented now -

onnxruntime/js/web/test/suite-test-list.jsonc

Line 1229 in 415c26e

// "test_squeeze_negative_axes",

?

good catch... since it's already merged , maybe we should fix it together with a similar implementation of Unsqueeze in another PR?

visheratin · 2023-05-27T02:04:48Z

Sorry, I missed the part with commented tests. Will fix with the Unsqueeze implementation this weekend.

@hariharans29

### Description This PR adds an implementation of the Squeeze operator to WebGPU JSEP. The implementation follows the [operator schema](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Unsqueeze). To implement the `Unsqueeze` operator in the same fashion as the `Squeeze`, I added the `ComputeOutputShape()` method to the `UnsqueezeBase` class and made some slight modifications. Please let me know if it is a bad idea and if I should move this method to the JS implementation. I also uncommented test case lines in the `suite-test-list.jsonc` file for both Squeeze and Unsqueeze operators following @hariharans29's [comment](#16024 (comment)). ### How was it tested 1. I created a model with only one operator: ```Python import onnx.helper node = onnx.helper.make_node( "Unsqueeze", inputs=["T", "axes"], outputs=["y"], ) graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 4, 5]), onnx.helper.make_tensor_value_info("axes", 7, [2])], [onnx.helper.make_tensor_value_info("y", 1, [3, 1, 4, 5, 1])]) onnx.save(onnx.helper.make_model(graph), "unsqueeze.onnx") ``` 2. I compiled the runtime using @fs-eire's [instructions](https://gist.github.com/fs-eire/a55b2c7e10a6864b9602c279b8b75dce). 3. I ran the test models in the browser using this minimal setup: ```HTML <html> <script src=".\dist\ort.webgpu.min.js"></script> <script> async function run() { const session = await ort.InferenceSession.create('unsqueeze.onnx', {executionProviders: ['webgpu']}); console.log(session); const input = new ort.Tensor('float32', new Float32Array(60), [3, 4, 5]); const dim = new ort.Tensor('int64', [1n, 4n], [2]); const output = await session.run({ "T": input, "axes": dim }); console.log(output); } run(); </script> </html> ``` ### Motivation and Context Improve operator coverage for WebGPU JSEP.

@fs-eire

### Description This PR adds an implementation of the `Squeeze` operator to WebGPU JSEP. The implementation follows the [operator schema](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Squeeze) and allows one or two inputs. ### How was it tested 1. I created two models. Without `axes`: ```Python import onnx.helper node = onnx.helper.make_node( "Squeeze", inputs=["T"], outputs=["y"], ) graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 1, 4, 5])], [onnx.helper.make_tensor_value_info("y", 1, [3, 4, 5])]) onnx.save(onnx.helper.make_model(graph), "squeeze.onnx") ``` And with `axes`: ```Python import onnx.helper node = onnx.helper.make_node( "Squeeze", inputs=["T", "axes"], outputs=["y"], ) graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 1, 4, 5]), onnx.helper.make_tensor_value_info("axes", 7, [1])], [onnx.helper.make_tensor_value_info("y", 1, [3, 4, 5])]) onnx.save(onnx.helper.make_model(graph), "squeeze-dim.onnx") ``` 2. I compiled the runtime using @fs-eire's [instructions](https://gist.github.com/fs-eire/a55b2c7e10a6864b9602c279b8b75dce). 3. I ran the test models in the browser using this minimal setup: ```HTML <html> <script src=".\dist\ort.webgpu.min.js"></script> <script> async function run() { const session = await ort.InferenceSession.create('squeeze-dim.onnx', {executionProviders: ['webgpu']}); console.log(session); const input = new ort.Tensor('float32', new Float32Array(60), [3, 1, 4, 5]); const dim = new ort.Tensor('int64', [-3n], [1]); const output = await session.run({ "T": input, "axes": dim }); console.log(output); } run(); </script> </html> ``` ### Motivation and Context Improve operator coverage for WebGPU JSEP.

Squeee operator implementation.

26b1696

fs-eire mentioned this pull request May 20, 2023

[Web] WebGPU supported operator tracking #15952

Closed

github-advanced-security AI found potential problems May 21, 2023

View reviewed changes

Comment thread onnxruntime/core/providers/js/operators/squeeze.h Fixed

fs-eire requested review from guschmue and satyajandhyala May 22, 2023 21:26

satyajandhyala added the platform:web issues related to ONNX Runtime web; typically submitted using template label May 23, 2023

satyajandhyala reviewed May 23, 2023

View reviewed changes

Comment thread onnxruntime/core/providers/js/js_execution_provider.cc Outdated

satyajandhyala reviewed May 23, 2023

View reviewed changes

Comment thread onnxruntime/core/providers/js/operators/squeeze.h Outdated

satyajandhyala reviewed May 23, 2023

View reviewed changes

Reuse SqueezeBase functionality.

519175d

visheratin requested a review from satyajandhyala May 24, 2023 04:35

Linter fix.

35c3caa

guschmue previously approved these changes May 25, 2023

View reviewed changes

visheratin added 2 commits May 25, 2023 21:25

Merge remote-tracking branch 'origin/main' into squeeze

ebc2a90

Added operator to the list.

690fa7a

visheratin dismissed guschmue’s stale review via 690fa7a May 26, 2023 01:28

guschmue self-requested a review May 26, 2023 19:11

guschmue approved these changes May 26, 2023

View reviewed changes

fs-eire approved these changes May 26, 2023

View reviewed changes

satyajandhyala merged commit 415c26e into microsoft:main May 26, 2023

visheratin mentioned this pull request May 28, 2023

[JS/WebGPU] Unsqueeze operator implementation #16138

Merged

Conversation

visheratin commented May 20, 2023

Description

How was it tested

Motivation and Context

Uh oh!

visheratin commented May 20, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

satyajandhyala left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guschmue commented May 25, 2023

Uh oh!

azure-pipelines Bot commented May 25, 2023

Uh oh!

guschmue commented May 25, 2023

Uh oh!

azure-pipelines Bot commented May 25, 2023

Uh oh!

visheratin commented May 25, 2023

Uh oh!

fs-eire commented May 26, 2023

Uh oh!

visheratin commented May 26, 2023

Uh oh!

fs-eire commented May 26, 2023

Uh oh!

visheratin commented May 26, 2023

Uh oh!

guschmue commented May 26, 2023

Uh oh!

guschmue commented May 26, 2023

Uh oh!

azure-pipelines Bot commented May 26, 2023

Uh oh!

azure-pipelines Bot commented May 26, 2023

Uh oh!

guschmue commented May 26, 2023

Uh oh!

hariharans29 commented May 27, 2023

Uh oh!

fs-eire commented May 27, 2023

Uh oh!

visheratin commented May 27, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

satyajandhyala left a comment •

edited

Loading