fix hyde rag pipeline streaming #339
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
❌ Changes requested.
- Reviewed the entire pull request up to b470570
- Looked at
431lines of code in12files - Took 1 minute and 39 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
0additional comments because they didn't meet confidence threshold of50%.
Workflow ID: wflow_QpOcNe35cNDJZzy6
Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. We'll respond in a few minutes. Learn more here.
| @@ -152,6 +152,7 @@ async def stream_rag_completion( | |||
| filters: Optional[Dict[str, Any]] = None, | |||
| settings: Optional[Dict[str, Any]] = None, | |||
| generation_config: Optional[Dict[str, Any]] = None, | |||
| timeout: int = 300 | |||
There was a problem hiding this comment.
Consider reducing the default timeout value for stream_rag_completion or making it configurable to prevent potential resource exhaustion in scenarios with many concurrent long-running requests.
iCUE-Solutions
pushed a commit
to DeweyLearn/DeweyLearnR2R
that referenced
this pull request
Jul 18, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This PR involves renaming and refactoring of several files, classes, and functions, including updates to handle streaming responses, include timeout parameters, and add new functions.
Key points:
config_pipeline.pytoconfigurable_pipeline.py.run_qna_client.pytoqna_rag_client.py.stream_rag_completionfunction inr2r/client/base.pyandr2r/examples/clients/qna_rag_client.pyto include a timeout parameter.transform_messagefunction inr2r/core/pipelines/rag.pyandr2r/pipelines/hyde/rag.pyto include ageneration_configparameter.run_streamfunction inr2r/pipelines/hyde/rag.pyto return a generator.rag_completionfunction inr2r/main/app.pyto handle streaming responses.HyDEPipelineclass inr2r/pipelines/hyde/rag.pyto include a_construct_joined_contextfunction.test_test_functionfunction inr2r/tests/end_to_end.pyto include a test for thetest_functionfunction in theR2RClientclass.Generated with ❤️ by ellipsis.dev