Skip to content

[WIP] Deno browser WASM/WebGPU support #1344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Conversation

xenova
Copy link
Collaborator

@xenova xenova commented Jun 20, 2025

An experimental branch/PR to test running Transformers.js in Deno's WASM and WebGPU runtimes. This is a bit hacky, but it works... at least for now!

You can test this out by building this PR, and hosting the js files locally.

import { pipeline } from "http://localhost:8080/dist/transformers.js";

const extractor = await pipeline(
    "feature-extraction", "Xenova/all-MiniLM-L6-v2",
    // { device: "webgpu" }, // <-- Enable WebGPU acceleration
);

// Compute sentence embeddings
const sentences = ["Hello world", "This is an example sentence"];
const output = await extractor(sentences, { pooling: "mean", normalize: true });
console.log(output.tolist());

NOTE: This is different to running with the native EP (#1306), which will enable WebGPU support in Deno, via ONNXRuntime's Native WebGPU EP implementation.

There are some bugs/limitations with the current runtime (based on wgpu), e.g.,

An uncaught WebGPU validation error was raised: 
Shader 'Gather' parsing error: the `f16` enable-extension is not yet supported
  ┌─ wgsl:1:8
  │
1 │ enable f16;
  │        ^^^ this enable-extension specifies standard functionality which is not yet implemented in Naga
  │
  = note: Let Naga maintainers know that you ran into this at <https://github.com/gfx-rs/wgpu/issues/4384>, so they can prioritize it!

: the `f16` enable-extension is not yet supported
An uncaught WebGPU validation error was raised: ShaderModule with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalid
An uncaught WebGPU validation error was raised: In a set_pipeline command: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalid
An uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalid
An uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalid

but WASM works well 👍

cc @crowlKats for viz

@xenova xenova marked this pull request as draft June 20, 2025 03:21
@@ -142,7 +142,7 @@ export const env = {
remoteHost: 'https://huggingface.co/',
remotePathTemplate: '{model}/resolve/{revision}/',

allowLocalModels: !(IS_BROWSER_ENV || IS_WEBWORKER_ENV),
allowLocalModels: !(IS_BROWSER_ENV || IS_WEBWORKER_ENV || IS_DENO_RUNTIME),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deno doesn't support relative URLs, so this is necessary... even if we have filesystem access

Comment on lines +63 to +70
.replace(
`"object"==typeof process&&"object"==typeof process.versions&&"string"==typeof process.versions.node&&"renderer"!=process.type`,
"false",
)
.replace(
`typeof globalThis.process?.versions?.node == 'string'`,
"false",
)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very hacky, but is a current workaround, otherwise we get errors related to worker_threads not being prefixed with node:

cc @fs-eire if you have a better idea.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


} else if (apis.IS_NODE_ENV) {
ONNX = ONNX_NODE.default ?? ONNX_NODE;
} else if (apis.IS_NODE_ENV && (ONNX = ONNX_NODE.default ?? ONNX_NODE)?.InferenceSession) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handles an edge case where it is a node-like environment (e.g., Deno), but we are running in an environment where onnxruntime-node is not bundled (but onnxruntime-wasm is)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants