Is response streaming supported?

[sundar7D0]

Since the overall delay in returning the response from ChatGPT along with the context can be huge, does the slack-bot support response streaming where the tokens are returned to user as and when they are returned from the ChatGPT API using 'stream=True' argument?

[gabrielkoo]

Since the overall delay in returning the response from ChatGPT along with the context can be huge, does the slack-bot support response streaming where the tokens are returned to user as and when they are returned from the ChatGPT API using 'stream=True' argument?

Yes. You can use chat.update.

1. Send a initial response and get the ts value of the message.
2. Set stream=True in the openai.Completion.create call
3. Loop through the response stream, call chat.update in each loop with the latest unfinished text response from OpenAI.

That’s it! Please feel free to create a pull request if you have done so, it will help others too!