Added WebNN Intro and Tutorial #20719
Conversation
|
|
||
| #### Does WebNN support other operating systems? | ||
|
|
||
| Currently, WebNN best supports the Windows operating system. A version for Mac operating systems is in progress. |
There was a problem hiding this comment.
More than just Mac are being written, including ChromeOS, and Android.
I think it's more accurate for the second sentence to simply say "Versions for other operating systems are in progress".
| // Set up environment. | ||
| ort.env.wasm.numThreads = 1; | ||
| ort.env.wasm.simd = true; | ||
| ort.env.wasm.proxy = true; |
There was a problem hiding this comment.
proxy is not required. Setting proxy to true will run WebNN in worker, which may bring some overhead.
| ort.env.wasm.simd = true; | ||
| ort.env.wasm.proxy = true; | ||
| ort.env.logLevel = "verbose"; | ||
| ort.env.debug = true; |
There was a problem hiding this comment.
ort.env.logLevel = "verbose";
ort.env.debug = true;
I think these two flags are not necessary, which are only available in ort-web debug build.
There was a problem hiding this comment.
We could comment them with a description:
// Uncomment for additional information in debug builds:
// ort.env.logLevel = "verbose";
// ort.env.debug = true;
There was a problem hiding this comment.
they are always supported but create lots of debug output.
// or if (debug) sounds right
| ```html | ||
| <script src="./main.js"></script> | ||
| <script src="imagenetClasses.js"></script> | ||
| <script src="https://cdn.jsdelivr.net/npm/onnxruntime-web@1.18.0-dev.20240311-5479124834/dist/ort.webgpu.min.js"></script> |
There was a problem hiding this comment.
ORT-Web 1.18.0 has been released, please use that one and use ort.all.min.js:
<script src="https://cdnjs.cloudflare.com/ajax/libs/onnxruntime-web/1.18.0/ort.all.min.js" integrity="sha512-MHOBvmgua1hQ3KTf3WRm+6UwgeulioSsl38T264/q8+zuL+Z0g0k9+VjvWC3lxAWtENRzUz4Bri9zKsZ/LRZ8w==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>|
|
||
| // Configure WebNN. | ||
| const executionProvider = "webnn"; // Other options: webgpu | ||
| const modelPath = "./mobilenetv2-7.onnx" |
There was a problem hiding this comment.
Missing ; at the end of line.
| const modelPath = "./mobilenetv2-7.onnx" | ||
| const options = { | ||
| executionProviders: [{ name: executionProvider, deviceType: "gpu", powerPreference: "default" }], | ||
| freeDimensionOverrides: {"batch": 1, "channels": 3, "height": 224, "width": 224} |
There was a problem hiding this comment.
We'd better comment to user that how to set freeDimensionOverrides, as the keys in this object should map to the real input dim names in the model.
There was a problem hiding this comment.
e.g. in my mobilenet model, the key is batch_size, so I only need to set freeDimensionOverrides: {"batch_size": 1}
|
|
||
| ```js | ||
| <script src="https://cdnjs.cloudflare.com/ajax/libs/jimp/0.22.12/jimp.min.js" integrity="sha512-8xrUum7qKj8xbiUrOzDEJL5uLjpSIMxVevAM5pvBroaxJnxJGFsKaohQPmlzQP8rEoAxrAujWttTnx3AMgGIww==" crossorigin="anonymous" referrerpolicy="no-referrer"></script> | ||
| <script src="https://cdn.jsdelivr.net/npm/lodash@4.17.21/lodash.min.js"></script> |
There was a problem hiding this comment.
Are these two files used for imagenetClassesTopK?
These may be unnecessary, you can refer to getTopClasses method in webnn-samples, which doesn't rely on additional libs.
// Get top 3 classes of labels from output buffer
function getTopClasses(buffer, labels) {
const probs = Array.from(buffer);
const indexes = probs.map((prob, index) => [prob, index]);
const sorted = indexes.sort((a, b) => {
if (a[0] === b[0]) {
return 0;
}
return a[0] < b[0] ? -1 : 1;
});
sorted.reverse();
const classes = [];
for (let i = 0; i < 3; ++i) {
const prob = sorted[i][0];
const index = sorted[i][1];
const c = {
label: labels[index],
prob: (prob * 100).toFixed(2),
};
classes.push(c);
}
return classes;
}
|
|
||
| The Web Neural Network (WebNN) API is an emerging web standard that allows web apps and frameworks to accelerate deep neural networks with GPUs, CPUs, or purpose-built AI accelerators such as NPUs. The WebNN API leverages the DirectML API on Windows to access the native hardware capabilities and optimize the execution of neural network models. | ||
|
|
||
| As the use of AI/ML in apps become more popular, the WebNN API provides the following benefits: |
There was a problem hiding this comment.
this paragraph should be removed since this applies to ort-web, not just webnn
| * *High Availability* - No reliance on the network after initial asset caching for offline case, as web apps and frameworks can run neural network models locally even when the internet connection is unavailable or unreliable. | ||
| * *Low Server Cost* - Computing on client devices means no servers needed, which helps web apps to reduce the operational and maintenance costs of running AI/ML services in the cloud. | ||
|
|
||
| AI/ML supported by WebNN include generative AI, person detection, face detection, semantic segmentation, skeleton detection, style transfer, super resolution, image captioning, machine translation, and noise suppression. |
There was a problem hiding this comment.
Since this is onnxruntime documentation you might want to use 'WebNN execution provider"
| AI/ML supported by WebNN include generative AI, person detection, face detection, semantic segmentation, skeleton detection, style transfer, super resolution, image captioning, machine translation, and noise suppression. | ||
|
|
||
| > [!NOTE] | ||
| > The WebNN API is still in progress, with GPU support in a preview state and NPU support coming soon. The WebNN API should not currently be used in a production environment. |
There was a problem hiding this comment.
I would replace 'WebNN API' with 'WebNN and WebNN execution provider'.
Maybe I'd avoid too much specifics like GPU and NPU since this will go stale and instead point to something like
https://webmachinelearning.github.io/webnn-status/
|
|
||
| ## Framework support | ||
|
|
||
| WebNN is designed as a backend API for web frameworks. For Windows, we recommend using [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/). This gives a familiar experience to using DirectML and ONNX Runtime natively so you can have a consistent experience deploying AI in ONNX format across web and native applications. |
There was a problem hiding this comment.
This is the onnxruntime documentation so this section can be removed.
| > [!NOTE] | ||
| > Chromium based browsers can currently support WebNN, but will depend on the individual browser's implementation status. | ||
|
|
||
| ## Model support |
There was a problem hiding this comment.
I'm not sure about this section and would replace it with:
Demos and Example applications
You can a set of demos and examples applications here:
https://microsoft.github.io/webnn-developer-preview/
| 2. You also need to add the `imageDataToTensor` function that is referenced above, which will render the loaded image into a tensor format that will work with our ONNX model. This is a more involved function, though it might seem familiar if you've worked with similar image classification apps before. For an extended explanation, you can view [this ONNX tutorial](https://onnxruntime.ai/docs/tutorials/web/classify-images-nextjs-github-template.html#imagehelperts). | ||
|
|
||
| ```js | ||
| function imageDataToTensor(image) { |
There was a problem hiding this comment.
| } | ||
| ``` | ||
|
|
||
| ## Step 4: Call WebNN |
There was a problem hiding this comment.
You are calling onnxruntime, not webnn
| ```js | ||
| async function runModel(preprocessedData) { | ||
| // Set up environment. | ||
| ort.env.wasm.numThreads = 1; |
There was a problem hiding this comment.
Some of ort.env is looked at only once before you create the first session and changes later are ignored so you want to not set this here ... should be a static or go into an init() function
| ort.env.wasm.simd = true; | ||
| ort.env.wasm.proxy = true; | ||
| ort.env.logLevel = "verbose"; | ||
| ort.env.debug = true; |
There was a problem hiding this comment.
they are always supported but create lots of debug output.
// or if (debug) sounds right
| const executionProvider = "webnn"; // Other options: webgpu | ||
| const modelPath = "./mobilenetv2-7.onnx" | ||
| const options = { | ||
| executionProviders: [{ name: executionProvider, deviceType: "gpu", powerPreference: "default" }], |
There was a problem hiding this comment.
this dictionary is specific to webnn so you can use { name: "webnn" ...
Description
Added WebNN intro and tutorial
Motivation and Context
We are about to roll out a dev preview of WebNN