Regarding continue generation using API

[nithinreddyyyyyy]

Describe the bug

I have been using OpenAI API key for long time. When I give some question to the model, call the API and it'll return the answer, sometimes it won't return the complete answer due to token issues.

Recently, I saw that ChatGPT has come up with new feature, continue generation, where if i click on that, it is continuing the generation in the same block. Below is the picture,

Is there any way to do that for the API code as well?

To Reproduce

1. Completion of the answer
2. Limitation of tokens

Code snippets

import openai

messages = [
    {"role": "system", "content": "You are a kind helpful assistant."},
]

while True:
    message = input("User : ")
    if message:
        messages.append(
            {"role": "user", "content": message},
        )
        chat = openai.ChatCompletion.create(
            model="gpt-3.5-turbo", messages=messages
        )
    
    reply = chat.choices[0].message.content
    print(f"ChatGPT: {reply}")
    messages.append({"role": "assistant", "content": reply})

OS

WindowsOS

Python version

Python 3.8.5

Library version

latest version

[zdwork]

Is there any solution to this problem?

[nithinreddyyyyyy]

Is there any solution to this problem?

Not sure, waiting for the reply

[DogAndCode]

Would like to know too!

[logankilpatrick]

Right now, you would have to manually take care of this by looking at the stop reason and passing some truncated context to the model and have it continue generating (which is more or less what is going on behind the scenes in ChatGPT). Agreed that it would be nice to take care of this for developers but it does not exist today so tagging this as feedback.