This repository was archived by the owner on Nov 21, 2024. It is now read-only.
Repository navigation
Support prefill only models
请移步 [RFC]: Support encode only models by Workflow Defined Engine
祝好
light-vllm v0.2.2
Warning
Not rigorously tested.
For research and experimentation only.
Use vllm for production environment