Skip to content

fix the max tokens for gpt-4-turbo 128K #374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 23, 2023
Merged

fix the max tokens for gpt-4-turbo 128K #374

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion service/src/chatgpt/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,12 @@ export async function initApi(key: KeyConfig, chatModel: string) {
// Set the token limits based on the model's type. This is because different models have different token limits.
// The token limit includes the token count from both the message array sent and the model response.
// 'gpt-35-turbo' has a limit of 4096 tokens, 'gpt-4' and 'gpt-4-32k' have limits of 8192 and 32768 tokens respectively.

// Check if the model type is GPT-4-turbo
if (model.toLowerCase().includes('1106-preview')) {
//If it's a '1106-preview' model, set the maxModelTokens to 131072
options.maxModelTokens = 131072
options.maxResponseTokens = 32768
}
// Check if the model type includes '16k'
if (model.toLowerCase().includes('16k')) {
// If it's a '16k' model, set the maxModelTokens to 16384 and maxResponseTokens to 4096
Expand Down