#
vllm
Here are 2 public repositories matching this topic...
【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架,支持dynamic batching、streaming模式,支持python/c++双语言,可限制,可拓展,高性能。帮助用户快速地将模型部署到线上,并通过http/rpc接口方式提供服务。
-
Updated
Sep 5, 2024 - C++
Improve this page
Add a description, image, and links to the vllm topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the vllm topic, visit your repo's landing page and select "manage topics."