Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Update n_parallel default value #227

Merged
merged 2 commits into from
Dec 3, 2023
Merged

Update n_parallel default value #227

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
47e7724
update n_paralel default value
hahuyhoang411 Dec 1, 2023
6e088df
update min max value for chatcompletion in yaml
hahuyhoang411 Dec 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ Table of parameters
| `ngl` | Integer | The number of GPU layers to use. |
| `ctx_len` | Integer | The context length for the model operations. |
| `embedding` | Boolean | Whether to use embedding in the model. |
| `n_parallel` | Integer | The number of parallel operations. Uses Drogon thread count if not set. |
| `n_parallel` | Integer | The number of parallel operations. |
| `cont_batching` | Boolean | Whether to use continuous batching. |
| `user_prompt` | String | The prompt to use for the user. |
| `ai_prompt` | String | The prompt to use for the AI assistant. |
Expand Down
10 changes: 7 additions & 3 deletions docs/openapi/NitroAPI.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ openapi: 3.0.0
info:
title: Nitro API
description: Please see https://nitro.jan.ai/ for documentation.
version: "0.1.8"
version: "0.1.19"
contact:
name: Nitro Discord
url: https://github.com/janhq/nitro
Expand Down Expand Up @@ -231,8 +231,8 @@ components:
description: Whether to use continuous batching.
n_parallel:
type: integer
default: Automatically set to Dragon threads
example: 4
default: 1
example: 1
nullable: true
description: The number of parallel operations. Only set when enable continuous batching.
cpu_threads:
Expand Down Expand Up @@ -563,11 +563,15 @@ components:
description: Defines specific tokens or phrases at which the model will stop generating further output
frequency_penalty:
type: number
min: 0
max: 2
default: 0
description: Adjusts the likelihood of the model repeating words or phrases in its output
presence_penalty:
type: number
default: 0
min: 0
max: 2
description: Influences the generation of new and varied concepts in the model's output
temperature:
type: number
Expand Down