This repository was archived by the owner on Nov 21, 2024. It is now read-only.
Repository navigation
light-vllm v0.1 Baseline
这个项目是个人实验项目
- 首先 fork from vllm v0.5.4
- 删除以下模块
- distributed ray
- adapter_commons prompt_adapter lora
- multimodal
- spec_decode guided_decoding
- async
- usage metrics tracing observability
性能基线
作为尝试自己一些想法基础看上去不错。
祝好
light-vllm v0.1
Warning
Not rigorously tested.
For research and experimentation only.
Use vllm for production environment