Skip to content

Commit

Permalink
Update text-generation.md
Browse files Browse the repository at this point in the history
Signed-off-by: Taikono-Himazin <kazu@po.harenet.ne.jp>
  • Loading branch information
Taikono-Himazin committed Apr 21, 2024
1 parent 284ad02 commit 2c4a778
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion docs/content/docs/features/text-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,10 @@ parameters:
# swap_space: 2
# Uncomment to specify the maximum length of a sequence (including prompt and output)
# max_model_len: 32768
# Uncomment and specify the number of Tensor divisions.
# Allows you to partition and run large models. Performance gains are limited.
# https://github.com/vllm-project/vllm/issues/1435
# tensor_parallel_size: 2
```

The backend will automatically download the required files in order to run the model.
Expand Down Expand Up @@ -356,4 +360,4 @@ template:
completion: |
{{.Input}}
```
```

0 comments on commit 2c4a778

Please sign in to comment.