Skip to content

feat(vllm serving): Define requirements for models to be served via vLLM #5

@christian-pinto

Description

@christian-pinto

Objective

The objective of this issue is that of defining a model's requirements for serving via vLLM, in addition to the dependencies resolution.
Some of the models will support serving via vLLM. If this is the case we first need to validate this integration before we can add the model to Algorithm Nexus.

Expected outcome

Upon completion of this issue the model requirements document defined in #2 is updated with the vLLM serving requirements, and the CONTRIBUTING_NEW_MODEL.md is updated accordingly.

Metadata

Metadata

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions