Skip to content

How to reduce token usage #390

@yaoye7758521

Description

@yaoye7758521

The usage fee for the text-davinci-3 model is $0.0200/1K tokens, which is very expensive.On average, around 3000 tokens are required per simple request,Each simple conversation approximately costs 0.1 US dollars,this is very economically unfriendly to users.
image
I am trying to use a more economical text ada-1 model.
self.llm = OpenAI(temperature=0,model_name="text-ada-001")
But the maxmum token size of this model is restricted to 2048,which lead to the following Error.
openai.error.InvalidRequestError: This model's maximum context length is 2049 tokens, however you requested 2074 tokens (1818 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.
So I want to ask if there is a way to reduce the number of tokens per request or split each long request in order to use a more economical ada model.
Looking forward to hearing from you,Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions