onnx2webnn

Exports the ONNX file to a WebNN JavaScript file and a bin file containing the weights.

This project is derived from onnx2json.

Setup

$ pip install -U onnx protobuf numpy

CLI Usage

usage: onnx2webnn.py [-h] -if INPUT_ONNX_FILE_PATH -oj OUTPUT_JS_PATH [-nhwc] [-json]
                     [-i JSON_INDENT] [-imagenet] [-segmentation]

options:
  -h, --help            show this help message and exit
  -if INPUT_ONNX_FILE_PATH, --input_onnx_file_path INPUT_ONNX_FILE_PATH
                        Input ONNX model path. (*.onnx)
  -oj OUTPUT_JS_PATH, --output_js_path OUTPUT_JS_PATH
                        Output WebNN JavaScript file path (*.js)
  -nhwc, --nhwc         Generate WebNN operators taking nhwc input layout, including conv2d, convTranspose2d, resample2d and pool2d
  -json, --dump_json    Dump the JSON representation of ONNX model
  -i JSON_INDENT, --json_indent JSON_INDENT
                        Number of indentations in JSON. (default=2)
  -imagenet, --imagenet
                        Test imagenet model in the generated index.html
  -segmentation, --segmentation
                        Test segmentation model in the generated index.html

Generate WebNN JavaScript model

Note: Before using this tool, please ensure override the free dimensions of input ONNX model by using onnxruntime_perf_test tool, for example

$ onnxruntime_perf_test -I -r 1 -u mobilenetv2-12-static.onnx -f batch_size:1 -o 1 mobilenetv2-12.onnx

Make a folder "mobilenet" that will contain the generated files:

$ mkdir mobilenet

Then run the following command to create WebNN JavaScript model for the static ONNX model:

$ python onnx2webnn.py -if ../sample_models/mobilenetv2-12-static.onnx -oj mobilenet/mobilenet.js

It will generate "mobilenet.bin" and "mobilenet.js" besides "mobilenet.json" in "mobilenet" folder.

An "index.html" is also generated for testing the WebNN model.

Start a node.js http-server in the folder containing generated model files and launch a web browser with URL http://localhost:8080/.

$ http-server

Generate NHWC WebNN model

The default input layout of ONNX model is NCHW, however some WebNN backends prefer to NHWC input layout, such as TFLite for CPU and GPU. For those backends, using WebNN NHWC model may have better performance.

To generate the NHWC WebNN model from an ONNX model, add the "--nhwc" switch, such as

$ python onnx2webnn.py -if ../sample_models/mobilenetv2-12-static.onnx -oj mobilenet_nhwc/mobilenet_nhwc.js -nhwc

WebNN API exposes the backend preferred layout via MLContext.opSupportLimit().preferredInputLayout, a web app can load the corresponding WebNN model based on the preferred layout. For example, in JavaScript code

const deviceType = 'gpu'; // or 'cpu', 'npu'
const context = await navigator.ml.createContext({deviceType});
const layout = context.opSupportLimits().preferredInputLayout;
let webnnModel;
if (layout == 'nhwc') {
    webnnModel = new MobilenetNhwc();
} else {
    webnnModel = new Mobilenet();
}
// Load the weights in preferred layout and build the graph.
await webnnModel.build({deviceType});
// Do inference with webnnModel.run()

Generate QDQ WebNN models

This tool supports converting QDQ (Quantize-Dequantize) ONNX model to WebNN model. According WebNN quantizeLinear and dequantizeLinear spec, it may need to reshape the scale and zero point tensor according to the rank of input tensor and axis. To support this feature, please ensure the ONNX model has shape info for each output tensor by running onnx-simplifier, e.g.

> pip3 install onnxsim
> onnxsim ../sample_models/mobilenetv2-12-qdq-static.onnx ../sample_models/mobilenetv2-12-qdq-static-simplified.onnx

After that, generate WebNN model with the following command line

> python onnx2webnn.py -if ../sample_models/mobilenetv2-12-qdq-static-simplified.onnx -oj mobilenet_qdq/mobilenet_qdq.js

For NHWC model, use

> python onnx2webnn.py -if ../sample_models/mobilenetv2-12-qdq-static-simplified.onnx -oj mobilenet_qdq_nhwc/mobilenet_qdq_nhwc.js -nhwc

Generate model with shape info

Some models contain nodes that need to know the shape info before conversion. For example squeezenet1.1-7.onnx, it contains Concat operator with axis -1. To handle it, please ensure the ONNX model has shape info for each output tensor by running onnx-simplifier, e.g.

> pip3 install onnxsim
> onnxsim ../sample_models/squeezenet1.1-7.onnx ../sample_models/squeezenet1.1-7-simplified.onnx

After that, generate WebNN model from the simplified ONNX model with the following command line

> python onnx2webnn.py -if ../sample_models/squeezenet1.1-7-simplified.onnx -oj squeezenet/squeezenet.js

Test image classification models

Models trained by ImageNet for image classification, such as MobileNet, can be tested with index.html generated with --imagenet switch. This is uesful to verify the correctness of generated WebNN models.

For example

$ python onnx2webnn.py -if ../sample_models/mobilenetv2-12-static.onnx -oj mobilenet/mobilenet.js -imagenet

It will generate image pre-processing and result post-processing code in index.html. After launching index.html, you can upload an image as input. After running the model, the top 5 classification results will be displayed.

Test image segmentation models

For image segmetnation models, such as MediaPipe selfie segmentation model, you can specify --segmentation switch to generate the testing code in index.html. This is useful to verify the correctness of generated WebNN models.

For example, download MediaPipe selfie segmentation ONNX model from https://huggingface.co/onnx-community/mediapipe_selfie_segmentation/blob/main/onnx/model.onnx, rename it to mediapipe-selfie-segmentation.onnx.

Use onnx2sim to override the free dimension and do the shape inference:

$ onnxsim mediapipe-selfie-segmentation.onnx mediapipe-selfie-segmentation-simplified.onnx --overwrite-input-shape 1,3,256,256

Generate the WebNN model with test code

$ python onnx2webnn.py -if ..\sample_models\mediapipe-selfie-segmentation-simplified.onnx -oj selfie_segmentation_nhwc\selfie_segmentation_nhwc.js -nhwc -segmentation

It will generate image pre-processing and result post-processing code in index.html. After launching index.html, you can upload an image as input. After running the model, the original image, mask and masked image will be displayed.

Dump JSON

You can also dump the JSON file for debugging purpose.

$ python onnx2webnn.py -if ../sample_models/mobilenetv2-12-static.onnx -oj mobilenet/mobilenet.js -json