Skip to content

use WebGPU EP instead of JSEP in WebAssembly #24856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 19, 2025

Conversation

fs-eire
Copy link
Contributor

@fs-eire fs-eire commented May 25, 2025

Description

This PR allows to use WebGPU EP in onnxruntime-web NPM package.

Migration Plan

Currently, there are 2 different EPs implemented the WebGPU backend of onnxruntime-web. They are JSEP and WebGPU EP. The migration plan is to replace the JSEP with WebGPU EP and eventually remove the JSEP. The plan contains the following stages:

  • STAGE 1: enable WebGPU EP on onnxruntime-web in local build. (Done)
  • STAGE 2: enable WebGPU EP on onnxruntime-web in the public package. (This PR)
  • STAGE 3: remove JSEP from onnxruntime-web.

Package consumption changes

  • Default import (import 'onnxruntime-web') and CPU only import (import 'onnxruntime-web/wasm') keeps their previous behaviors.

  • WebGPU import (import 'onnxruntime-web/webgpu') will now use WebGPU EP instead of JSEP. Previously it was the same as default import.

    • WebGPU import will use a different suffix for the .mjs and .wasm name (which was .jsep):

      • ort-wasm-simd-threaded.asyncify.mjs
      • ort-wasm-simd-threaded.asyncify.wasm
    • The suffix .asyncify is used as .jspi is planned for future. They are 2 different ways of emscripten's async implementation (sync C++ function calls async JS function)

@fs-eire fs-eire force-pushed the emsdk-webgpu branch 4 times, most recently from 3ae663c to 2e67062 Compare June 2, 2025 20:57
@fs-eire fs-eire changed the title [WIP] use WebGPU EP instead of JSEP in WebAssembly use WebGPU EP instead of JSEP in WebAssembly Jun 2, 2025
@fs-eire fs-eire marked this pull request as ready for review June 2, 2025 21:12
@fs-eire fs-eire force-pushed the emsdk-webgpu branch 2 times, most recently from c59022b to 414a928 Compare June 3, 2025 00:01
@fs-eire
Copy link
Contributor Author

fs-eire commented Jun 11, 2025

TODO: resolve conflicts after #25022 is merged.

guschmue
guschmue previously approved these changes Jun 17, 2025
@guschmue
Copy link
Contributor

both jsep and webgpu-ep builds for for me.

@fs-eire
Copy link
Contributor Author

fs-eire commented Jun 17, 2025

@Honry could you please help to test/validate if this change breaks WebNN?

This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

@Honry
Copy link
Contributor

Honry commented Jun 18, 2025

@Honry could you please help to test/validate if this change breaks WebNN?

This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

@Honry
Copy link
Contributor

Honry commented Jun 18, 2025

@Honry could you please help to test/validate if this change breaks WebNN?
This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

@fs-eire, both work! BTW, will we have a build specific for WebNN? e.g. ort.webnn.min.js, onnxruntime-web/webnn

@fs-eire
Copy link
Contributor Author

fs-eire commented Jun 18, 2025

@Honry could you please help to test/validate if this change breaks WebNN?
This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

@fs-eire, both work! BTW, will we have a build specific for WebNN? e.g. ort.webnn.min.js, onnxruntime-web/webnn

Will not. This is the plan:

currently both onnxruntime-web and onnxruntime-web/webgpu supports WebNN.
we keep both for testing and validation purpose.

once we think the time is mature, we will use WebGPU EP in onnxruntime-web and onnxruntime-web/webgpu will be deprecated. eventually everyone should use onnxruntime-web (WebGPU EP + WebNN EP) and there is going to be no JSEP.

@Honry
Copy link
Contributor

Honry commented Jun 18, 2025

@Honry could you please help to test/validate if this change breaks WebNN?
This change is expected to have no behavior change for WebNN. Specifically:

  • importing from "onnxruntime-web" (import * as ort from 'onnxruntime-web';) should be using the old JSEP + WebNN EP.
  • importing from "onnxruntime-web/webgpu" (import * as ort from 'onnxruntime-web/webgpu';) should be using the new WebGPU EP + WebNN EP

Both should work for WebNN

Sure, I will verify it with some demos.

@fs-eire, both work! BTW, will we have a build specific for WebNN? e.g. ort.webnn.min.js, onnxruntime-web/webnn

Will not. This is the plan:

currently both onnxruntime-web and onnxruntime-web/webgpu supports WebNN. we keep both for testing and validation purpose.

once we think the time is mature, we will use WebGPU EP in onnxruntime-web and onnxruntime-web/webgpu will be deprecated. eventually everyone should use onnxruntime-web (WebGPU EP + WebNN EP) and there is going to be no JSEP.

That make sense, thank you for the explanation!

@fs-eire fs-eire merged commit 2c041e0 into microsoft:main Jun 19, 2025
85 of 88 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants