Proposed use cases from Web LLM project

[anssiko]

From @tqchen in #375 (comment):

Great to see this thread, as part of of WebLLM project https://github.com/mlc-ai/web-llm We are also building related compilation flows for WebGPU, with ability to run llama up to 70b(with latest M2 max) https://webllm.mlc.ai/

There are great synergies to webnn related projects that possibly enables future hybrid executions of models(e.g. webgpu for customized op and some through webnn)

Discussed: WebML WG Teleconference – 16 November 2023

Related WebNN issue: custom operations #6

[tqchen]

Sorry for getting to this late! Happy to participate in future discussions! I think one quite useful feature is to enable a json schema of the webnn declaration, so that the compiler projects can generate them and invoke executions without explicitly doing so in javascript.

[anssiko]

It was my pleasure to host a discussion with @tqchen on the WebML WG's teleconference today. The minutes are at: https://www.w3.org/2024/01/11-webmachinelearning-minutes.html#t01

(Any errors in the minutes are entirely mine.)

We recorded the following as the top thing to help the Web LLM and related projects adopt WebNN:

1. Experiment with building WebNN TVM backend

@tqchen feel free to use this issue to drop any pointers to relevant references or note additional things the group could experiment with.

Thanks for the great work and presentation! I look forward to a continued productive collaboration.

[anssiko]

@tqchen thanks again for joining our meeting and presenting to us the exciting work you and your team has been doing with Web LLM, MLC and other projects.

A bunch of group participants were interested in revisiting the material you presented. If you have that available online feel free to drop a link here.

@fdwr feel free to use this issue to ask questions from @tqchen related to the discussion we had. I propose we use this issue to document areas to explore in this space to be discussed on our future calls, TVM backend being one.

[anssiko]

Folks interested in Web LLM and related projects should check the public Machine Learning Compilation course. The video lectures and notes are high quality.

Included are also code labs for e.g. translating low-level NumPy to TensorIR, a tensor program abstraction in Apache TVM. This will help build mental models on how WebNN API could plug in as another backend.