feat: Support WasmEdge and its ggml plugin on ARM NPU

[alabulei1]

Summary

One of the advantages of using WasmEdge as the LLM inference runtime is that WasmEdge is portable across different CPUs and GPUs. So it's important to support more chips for WasmEdge.

ARM NPU chip is a popular AI processor that WasmEdge should support.

Details

Support running LLM inference with WasmEdge on ARM NPU

Appendix

No response

[Wck-iipi]

I would like to work on this issue. If you can guide me with references I would be grateful thanks.

[hangedfish]

I think we can start with Rockchip RK3588 SOC, which is a popular chip recently. It supports 32GB of memory, which is enough for LLM. There are also a large number of SBC (Single-board computers) products that can be tested, such as Radxa ROCK 5B/5C and Orange Pi 5 Plus.

https://github.com/airockchip/rknn-toolkit2

Good luck