Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add StreamingResponse support for ConversationalRetrievalChain chain type #16

Merged
merged 10 commits into from
May 10, 2023

Conversation

rogalvil
Copy link
Contributor

@rogalvil rogalvil commented May 2, 2023

Add support to Conversational Retrieval

But I not know how implement websocket example

Needs to update to version 0.0.157

Possibly it will not be necessary to create a custom response and it may work with RetrievalQAStreamingResponse

@rogalvil rogalvil marked this pull request as ready for review May 2, 2023 23:24
@ajndkr ajndkr changed the title Conversational Retrieval feat: add support for ConversationalRetrievalChain chain type May 3, 2023
@ajndkr
Copy link
Owner

ajndkr commented May 3, 2023

will check this out in a few hours! nice to see you actively contributing here! 😄

@ajndkr ajndkr added the feature New feature or request label May 3, 2023
@rogalvil rogalvil marked this pull request as draft May 3, 2023 18:37
@ajndkr
Copy link
Owner

ajndkr commented May 3, 2023

@rogalvil i think the solution revolves around this line: https://github.com/ajndkr/fastapi-async-langchain/blob/3266a94e8bdf2e24b2a0dc7f0b73b2322f04d099/fastapi_async_langchain/websockets/base.py#L30

instead of receive_text(), we should use receive() and detect if the message is text or json.

@audvin
Copy link

audvin commented May 10, 2023

Question on this PR, is it functional for http but just holding for web socket support?

@ajndkr
Copy link
Owner

ajndkr commented May 10, 2023

Question on this PR, is it functional for http but just holding for web socket support?

that's correct. I will take over this PR and update it to limit the update to http for now. we can add the websocket support later.

@ajndkr ajndkr changed the title feat: add support for ConversationalRetrievalChain chain type feat: add StreamingResponse support for ConversationalRetrievalChain chain type May 10, 2023
@ajndkr ajndkr marked this pull request as ready for review May 10, 2023 20:18
@ajndkr
Copy link
Owner

ajndkr commented May 10, 2023

example curl request:

```bash
curl -N -X POST \
-H "Accept: text/event-stream" -H "Content-Type: application/json" \
-d '{
    "query": "Give me a code sample",
    "history": [
        [
            "What is a Text Splitter?",
            "Text Splitter is a module that is responsible for breaking up a document into smaller pieces, or chunks, that can be more easily processed."
        ],
        [
            "List all text splitter supported",
            "Langchain provides several different text splitters to help with processing text data. These include the Character Text Splitter, Hugging Face Length Function, Latex Text Splitter, Markdown Text Splitter, NLTK Text Splitter, Python Code Text Splitter, RecursiveCharacterTextSplitter, Spacy Text Splitter, tiktoken (OpenAI) Length Function, and TiktokenText Splitter."
        ]
    ]
}' \
http://localhost:8000/chat

Will add this later to repo docs.

@ajndkr ajndkr merged commit b694c68 into ajndkr:main May 10, 2023
@rogalvil rogalvil deleted the feature/conversational-retrieval branch May 11, 2023 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants