WebGL support

[Macoron]

Whisper.cpp supports webasm and even has some online demos running into browser.

I didn't research that, but it seems that Unity support native C++ plugins for WebGL builds. So in theory, you can use cmake to compile whisper.cpp into .bc and embed it into web unity build.

Also Unity doesn't support microphone input for WebGL out of the box and you need third-party plugin to record anything.

[Macoron]

Did some tests today. Here is rough list of tasks that need to be done:

• Unity doesn't support C# multithreading. Whisper blocks main unity thread and app hangs until inference complete. The good news: Unity 2022.2+ supports C++ multithreading. If I add parallel job creation in whisper.cpp code - this should work.
• Bump Unity project to 2022 LTS
• Inference time is very slow comparing with official whisper.cpp demo. Unity transcribe "jfk.wav" for 15 seconds, while official demo works for 3 seconds. Researching this.
• No microphone support. Need to look for some MIT license compatible microphone library for Unity.

If you want to play around with it, check webgl branch.

[Bodekaer]

Hi @Macoron, any updates on webgl support. Did you get it running?

[Macoron]

Not much, biggest blocker is that Unity doesn't support C# multithreading. But you can checkout webgl branch. It's a bit outdated, but still working.

[D4N005H]

Not much, biggest blocker is that Unity doesn't support C# multithreading. But you can checkout webgl branch. It's a bit outdated, but still working.

Based on this source, Async/Await is possible for WebGL with this addon: https://github.com/Cysharp/UniTask