How do I do a Streming Response?

[DiogenesBR]

Hi,

I'm trying to implement a streaming response using

from streamlit_chat import message

Like in this blog post
https://medium.com/@avra42/how-to-stream-output-in-chatgpt-style-while-using-openai-completion-method-b90331c15e85

But I don't know if its possible.

In Gradio Chatbot you need to keep updating the same message.
https://gradio.app/creating-a-chatbot/#add-streaming-to-your-chatbot

[Ajaypawar02]

I also had the same query , is there any update regarding this

[sfc-gh-jcarroll]

Yes, it's possible. You can use the native Chat UI instead of installing streamlit_chat custom component. Here's an example:

import openai
import streamlit as st

# Initialize the chat messages history
openai.api_key = st.secrets.OPENAI_API_KEY
if "messages" not in st.session_state:
    st.session_state.messages = [{"role": "assistant", "content": "How can I help you?"}]

# Prompt for user input and save
if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})

# display the existing chat messages
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.write(message["content"])

# If last message is not from assistant, we need to generate a new response
if st.session_state.messages[-1]["role"] != "assistant":
    with st.chat_message("assistant"):
        response = ""
        resp_container = st.empty()
        for delta in openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=st.session_state.messages,
            stream=True,
        ):
            response += delta.choices[0].delta.get("content", "")
            resp_container.markdown(response)

        st.session_state.messages.append({"role": "assistant", "content": response})

Hope it helps!