Skip to content

Commit

Permalink
Update python/huggingfaceserver/README.md
Browse files Browse the repository at this point in the history
Co-authored-by: Filippe Spolti <filippespolti@gmail.com>
Signed-off-by: Alexa Griffith  <agriffith50@bloomberg.net>
  • Loading branch information
alexagriffith and spolti committed May 10, 2024
1 parent b196e98 commit 4a22f6c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion python/huggingfaceserver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ spec:
cpu: 100m
memory: 2Gi
```
3. Serve the llama2 model using huggingface vllm runtime. For the llama2 model, vllm is supported and used as the default model. If available for a model, vllm is set as the default runtime. Note - Model need to be supported by vllm otherwise KServe python runtime will be used as a failsafe.
3. Serve the llama2 model using huggingface vLLM runtime. For the llama2 model, vLLM is supported and used as the default model. If available for a model, vLLM is set as the default runtime. Note - Model needs to be backed by vLLM otherwise KServe python runtime will be used as a failsafe.
vllm supported models - https://docs.vllm.ai/en/latest/models/supported_models.html
```yaml
apiVersion: serving.kserve.io/v1beta1
Expand Down

0 comments on commit 4a22f6c

Please sign in to comment.