Can trained models run in browsers, just for model inference? #3100
-
|
Can trained models be loaded in Chrome browsers like the usage of node.js? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
It's not easy. While node.js can directly bind with C/C++ and execute x86_64 binaries, browsers can only execute WebAssembly (Wasm). While there are successful examples for small packages like NumPy, the upstream TensorFlow C++ library is too hard to build. xref: pyodide/pyodide#50 Another resolution is to use tf.js, which does not support all OPs or customized OPs. At least, one needs to write customized OPs in the JS way. Contribution is welcome if anyone is willing to spend time on it. |
Beta Was this translation helpful? Give feedback.
It's not easy.
While node.js can directly bind with C/C++ and execute x86_64 binaries, browsers can only execute WebAssembly (Wasm). While there are successful examples for small packages like NumPy, the upstream TensorFlow C++ library is too hard to build. xref: pyodide/pyodide#50
Another resolution is to use tf.js, which does not support all OPs or customized OPs. At least, one needs to write customized OPs in the JS way. Contribution is welcome if anyone is willing to spend time on it.