feat: LLM inference libraries support plan

[hydai]

Summary

There are various LLM inference libraries. WasmEdge already integrated llama.cpp, but we want to bring more to the community.

Details

Already supported:

1. PyTorch
2. TFLite
3. OpenVINO
4. llama.cpp

The support priority list:

• Tier 1:
  • burn-rs
• Tier 2:
  • Intel extension for transformers
  • whisper.cpp
  • RWKV
• Tier 3:
  • vllm
  • CTranslate2
  • candle
  • mlx

Please feel free to add any comments and suggestions. We would like to hear the voice of the community.
Also, if you are interested in contributing the new LLM inference libraries, please tell and show to us :-)

Happy new year!

Appendix

No response

[katopz]

Cool, i'm now playing with

• For RAG: llm-chain // just support Qdrant
• For mac: mlx // still in heavy development
• For fun: candle // infer tps seem to be a lot slower than llama.cpp atm but it will keep up somehow i hope.
• For co-pilot: Tabby // Support repos lookup but the output not satisfy me yet