Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use streaming to improve user experience #5

Closed
stoerr opened this issue Jun 21, 2023 · 0 comments · Fixed by #7
Closed

Use streaming to improve user experience #5

stoerr opened this issue Jun 21, 2023 · 0 comments · Fixed by #7
Assignees
@stoerr
Labels
enhancement New feature or request

Comments

@stoerr
Copy link
Member

stoerr commented Jun 21, 2023 •
edited

Unfortunately, ChatGPT is quite slow, so the user has to wait seconds to even tens of seconds for the response to appear. Therefore it is a must that we use the streaming API with Server-Sent Events (SSE)
https://github.com/openai/openai-cookbook/blob/main/examples/How_to_stream_completions.ipynb
https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format
so that the user can already see parts of the response while it's generating.
@stoerr stoerr self-assigned this Jun 21, 2023
@stoerr stoerr transferred this issue from ist-dresden/composum-launch Jun 21, 2023
@stoerr stoerr added the enhancement New feature or request label Jun 21, 2023
stoerr added a commit that referenced this issue Jun 21, 2023
@stoerr
#5 rewrite backend part of implementation to use Flow.Subscriber
2bde058
stoerr added a commit that referenced this issue Jun 21, 2023
@stoerr
#5 partial reimplementation of streaming; basics work but errors not.
340a5ad
stoerr added a commit that referenced this issue Jun 21, 2023
@stoerr
#5 fix error handling for base streaming impl
e9ed86f
stoerr added a commit that referenced this issue Jun 22, 2023
@stoerr
#5 part of EventStream implementation
4c8fb0b
stoerr added a commit that referenced this issue Jun 22, 2023
@stoerr
#5 backend implementation of streaming for translation
ddeefdc
stoerr added a commit that referenced this issue Jun 22, 2023
@stoerr
#5 remove obsolete chatgpt deletion filters
f9951ca
stoerr added a commit that referenced this issue Jun 23, 2023
@stoerr
#5 continue streaming implementation - StringstreamSlowdown
6081672
stoerr added a commit that referenced this issue Jun 23, 2023
@stoerr
#5 integrate StringstreamSlowdown int EventStream
8c23786
stoerr added a commit that referenced this issue Jun 23, 2023
@stoerr
#5 debugging
8fc7577
stoerr added a commit that referenced this issue Jun 23, 2023
@stoerr
#5 front end changes in translation dialog for streaming
74d6dba
stoerr added a commit that referenced this issue Jun 23, 2023
@stoerr
#5 fix character encoding
12006d5
@stoerr stoerr linked a pull request Jun 23, 2023 that will close this issue
Feature/5 use streaming to improve user experience #7
Merged
@stoerr stoerr mentioned this issue Jun 23, 2023
Closed
stoerr added a commit that referenced this issue Jun 26, 2023
@stoerr
#5 streaming for content creation
fd50e04
stoerr added a commit that referenced this issue Jun 27, 2023
@stoerr
#5 create dialog: fix button enabling / disabling depending on dialog…
78b02b1 
… state
stoerr added a commit that referenced this issue Jun 27, 2023
@stoerr
#5 various fixes of creation / translation
adbfaa4
stoerr added a commit that referenced this issue Jun 27, 2023
@stoerr
#5 (almost) force HTML for create when richText
00dfbe6
stoerr added a commit that referenced this issue Jun 27, 2023
@stoerr
#5 use maxtokens instead of maxwords and use tokenization library for…
c948a67 
… shortening
@stoerr stoerr closed this as completed in #7 Jun 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stoerr stoerr

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Feature/5 use streaming to improve user experience ist-dresden/composum-AI
1 participant
@stoerr