Uncaught (in promise) Error: no available backend found. ERR: [webgpu] TypeError: Failed to fetch dynamically imported module:

[ucalyptus2]

System Info

Chrome v124

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

backend-impl.ts:137 Uncaught (in promise) Error: no available backend found. ERR: [webgpu] TypeError: Failed to fetch dynamically imported module: http://localhost:5173/ort-wasm-simd-threaded.jsep.mjs at pt (http://localhost:5173/node_modules/.vite/deps/@xenova_transformers.js?v=61f84580:1386:13) at async e.create (http://localhost:5173/node_modules/.vite/deps/@xenova_transformers.js?v=61f84580:1906:20) at async createInferenceSession (http://localhost:5173/node_modules/.vite/deps/@xenova_transformers.js?v=61f84580:9952:10) at async constructSessions (http://localhost:5173/node_modules/.vite/deps/@xenova_transformers.js?v=61f84580:17549:21) at async Promise.all (index 0) at async Phi3ForCausalLM.from_pretrained (http://localhost:5173/node_modules/.vite/deps/@xenova_transformers.js?v=61f84580:17847:14) at async AutoModelForCausalLM.from_pretrained (http://localhost:5173/node_modules/.vite/deps/@xenova_transformers.js?v=61f84580:20680:14) at async Promise.all (index 1) at async load (http://localhost:5173/src/worker.js?worker_file&type=module:137:32)

  | pt | @ | backend-impl.ts:137 -- | -- | -- | --   | Show less



Reproduction

npm run build && npm run dev on the webgpu-chat example directory in v3 branch

[ucalyptus2]

I can use your webgpu hf space (see below) but not the webgpu-chat example so I think Chrome and WebGPU are not enemies on my system.

[ucalyptus2]

Does #748 solve it ? I mean do I wait for v3 PR to merge for this to work ?
cc: @xenova

[xenova]

As already mentioned in this comment, this will only work when onnxruntime-web 1.18.0 releases.

[ucalyptus2]

@xenova also is it possible to install the onnxruntime-web from https://github.com/microsoft/onnxruntime/tree/rel-1.18.0 directly instead?

[ucalyptus2]

@xenova I still get the error coz https://cdn.jsdelivr.net/npm/onnxruntime-web@1.18.0-dev.20240430-204f1f59b9/dist/ort-wasm-simd-threaded.jsep.mjs doesn't exist. See screenshot below:

[ucalyptus2]

@xenova tried the same with 1.19.0-dev

https://cdn.jsdelivr.net/npm/onnxruntime-web@1.19.0-dev.20240509-69cfcba38a/dist/ort-wasm-simd-threaded.jsep.mjs does not exist

I am a noob at javascript so I don't know how these .mjs exist

[ucalyptus2]

@xenova can't run this example as well:
https://github.com/microsoft/onnxruntime-inference-examples/tree/main/js/chat

I don't get it. I can easily run your huggingface spaces, then why not this?
https://huggingface.co/spaces/Xenova/experimental-phi3-webgpu

[xenova]

We recently updated the dependencies, so if you pull the latest changes from v3, delete your vite cache and try run it again, it should work.

[congphuong]

This is workaround:
env.backends.onnx.wasm.wasmPaths = "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.19.0-esmtest.20240513-a16cd2bd21/dist/";

[ucalyptus2]

Thanks @congphuong , you are a fuckin life saver!!!!!!!

[trtm]

@ucalyptus2 @congphuong Where did you made this changes? Thanks!