This repository was archived by the owner on Nov 21, 2024. It is now read-only.
Repository navigation
Support encode only models
支持 xlm-roberta、bge-m3
Flash Attention 比较快
double buffer 加速并不明显
有意思
light-vllm v0.2.1
Warning
Not rigorously tested.
For research and experimentation only.
Use vllm for production environment