Streaming responses from OpenAI and GPT4All

[raulraja]

These changes introduce the capability of streaming in chat responses. It adds the following changes:

• A new function, createChatCompletions, has been added to the Chat interface, which allows creating chat completions based on a ChatCompletionRequest. This function returns a Flow<ChatCompletionChunk> representing the generated chat completions.
• The Chat interface now includes two overloaded versions of the promptStreaming function. These functions enable streaming by returning a Flow<String> that emits the generated chat responses as they become available.
• The implementation of createChatCompletions in GPT4All utilizes a Flow and channels to enable streaming of chat completions.
• The MockOpenAIClient has been updated to throw a NotImplementedError for chatCompletions since it's not implemented in the mock.
• The OpenAIClient implementation has been updated to utilize the OpenAI API's chat completion functionality and transform the response into the corresponding domain models (ChatCompletionChunk, ChatChunk, ChatDelta, etc.).

Example

package com.xebia.functional.xef.auto.streaming

import com.xebia.functional.xef.auto.llm.openai.OpenAI
import com.xebia.functional.xef.auto.llm.openai.OpenAIEmbeddings
import com.xebia.functional.xef.llm.Chat
import com.xebia.functional.xef.vectorstores.LocalVectorStore

suspend fun main() {
  val chat: Chat = OpenAI.DEFAULT_CHAT
  val embeddings = OpenAIEmbeddings(OpenAI.DEFAULT_EMBEDDING)
  val vectorStore = LocalVectorStore(embeddings)
  chat.promptStreaming(
    question = "What is the meaning of life?",
    context = vectorStore
  ).collect {
    print(it)
  }
}

These changes enable developers to perform streaming chat completions and receive responses incrementally, enhancing the real-time interactive chat experience.

[raulraja]

@xebia-functional/team-ai

[serras]

Should we maybe split these methods in a different interface? I see more and more that a client implements the interface only partially.

[raulraja]

They've already been split into its own interface in main inside Chat, ChatWithFunctions etc. The remaining place where it implements all of them like this is the MockClient. The AIClient interface in the main is already unused and should be removed. I'll push a commit to this PR to remove it.