feat: add streaming and websocket support for all chain types

[ajndkr]

After #6, it is evident that the chain_wrapper_fn logic is not generic for all chain types. There needs to be a Response class for each chain type. This issue will track progress.

Todo:

Streaming

• ConversationChain
• LLMChain
• LLMBashChain
• LLMCheckerChain
• LLMSummarizationCheckerChain
• LLMMathChain
• PALChain
• QAWithSourcesChain
• SQLDatabaseChain
• SequentialChain
• SimpleSequentialChain
• VectorDBQA
• VectorDBQAWithSourcesChain
• APIChain
• LLMRequestsChain
• TransformChain
• MapReduceChain
• OpenAIModerationChain
• SQLDatabaseSequentialChain
• AnalyzeDocumentChain
• HypotheticalDocumentEmbedder
• ChatVectorDBChain
• GraphQAChain
• ConstitutionalChain
• QAGenerationChain
• RetrievalQA
• RetrievalQAWithSourcesChain
• ConversationalRetrievalChain
• OpenAPIEndpointChain

Websocket

• ConversationChain
• LLMChain
• LLMBashChain
• LLMCheckerChain
• LLMSummarizationCheckerChain
• LLMMathChain
• PALChain
• QAWithSourcesChain
• SQLDatabaseChain
• SequentialChain
• SimpleSequentialChain
• VectorDBQA
• VectorDBQAWithSourcesChain
• APIChain
• LLMRequestsChain
• TransformChain
• MapReduceChain
• OpenAIModerationChain
• SQLDatabaseSequentialChain
• AnalyzeDocumentChain
• HypotheticalDocumentEmbedder
• ChatVectorDBChain
• GraphQAChain
• ConstitutionalChain
• QAGenerationChain
• RetrievalQA
• RetrievalQAWithSourcesChain
• ConversationalRetrievalChain
• OpenAPIEndpointChain

[pors]

Amazing project!

My use case would be an LLM chatbot, which requires state. FastAPI supports WebSockets which might be a good solution here, or is there another approach possible?

[ajndkr]

hi! i think it depends on how you are building the frontend but I do like the idea of adding support for websockets.

It's already implemented in https://github.com/hwchase17/chat-langchain but it's better if the implementation can be added to this package as well. I will create a feature request for this.

Would you like to contribute towards it?

[pors]

hi! i think it depends on how you are building the frontend

You mean by keeping track of history in the client? And thus sending it again at every request?

but I do like the idea of adding support for websockets.

It's already implemented in https://github.com/hwchase17/chat-langchain but it's better if the implementation can be added to this package as well. I will create a feature request for this.

Cool!

Would you like to contribute towards it?

I'll have a look if I am able to. Maybe you can point out where in your code it would fit best?

[ajndkr]

You mean by keeping track of history in the client? And thus sending it again at every request?

yup yup. for example: https://github.com/mckaywrigley/chatbot-ui.

I'll have a look if I am able to. Maybe you can point out where in your code it would fit best?

Yes sure. I think the responses module seems best for this: https://github.com/ajndkr/fastapi-async-langchain/tree/main/fastapi_async_langchain/responses. You can do the following:

• create base websocket response
• create base callback class for websocket inside callbacks module
• create websocket response for LLMChain (I can take over the rest)

[pors]

@ajndkr Sorry, but I don't have enough time for it. If I was a bit more familiar with FastAPI I'd do it, but I have only worked with Flask for APIs.

[ajndkr]

no problem! i will look into this. will let you know once it's ready.

[pors]

Great, thanks man!

[pors]

@ajndkr can I contact you somehow? I'm struggling with something similar to your project, maybe you can help me out.

[ajndkr]

@pors you can find me on twitter! https://twitter.com/ajndkr

[AmineDjeghri]

Using LLMChain doesn't work.

File "/home/outscale/miniconda3/envs/react-qback/lib/python3.9/site-packages/lanarky/routing/utils.py", line 88, in create_request_from_langchain_dependency
    raise TypeError(
TypeError: Error! Creating a Request model for 'LLMChain' is not currently supported by 'LangchainRouter.add_langchain_api_route()'.
Available chain types: ['ConversationChain', 'AgentExecutor', 'RetrievalQAWithSourcesChain', 'ConversationalRetrievalChain']

To use a custom chain type, you must define your own FastAPI endpoint.

I'm looking into a way to use a chat system but without using the memory since the history of messages is stored inside a database and there are different users using the chatbot.
So i can't use ConversationChain. I'm trying with LLMChain instead but it looks like it is not supported

[AmineDjeghri]

i added 'LLMChain' in Base Models in /home/outscale/miniconda3/envs/react-qback/lib/python3.9/site-packages/lanarky/routing/utils.py and it worked

['LLMChain', 'ConversationChain', 'AgentExecutor', 'RetrievalQAWithSourcesChain', 'ConversationalRetrievalChain']

[ajndkr]

good catch! @AmineDjeghri

[ajndkr]

closing this issue as I currently have no future plans to add support for all chain and agent types. I will add support iteratively based on demand/popularity.