feat(gateway): websocket based streaming and client #1608
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jina-bot
added
size/M
area/core
Latency summaryCurrent PR yields:
Breakdown
Backed by latency-tracking. Further commits will update this comment. |
jina-bot
added
size/L
area/testing
Codecov Report
@@ Coverage Diff @@
## master #1608 +/- ##
==========================================
- Coverage 84.82% 75.03% -9.79%
==========================================
Files 126 128 +2
Lines 6589 6713 +124
==========================================
- Hits 5589 5037 -552
- Misses 1000 1676 +676
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
jina-bot
added
area/cicd
|
|
||
| async def decode(self, websocket: WebSocket, message: Message) -> Any: | ||
| if 'text' in message or 'json' in message: | ||
| self.client_encoding = 'text' |
There was a problem hiding this comment.
this could be a good test case to cover
|
im taking this branch |
# Conflicts: # .github/workflows/ci.yml # extra-requirements.txt # tests/unit/flow/test_asyncflow.py
jina-bot
added
size/XL
component/docker
component/executor
component/type
executor/indexer
executor/meta
area/daemon
and removed
size/L
labels
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Recreated duplicate of #1551
Gateway
Added a
StreamingEndpoint(WS /stream) toRESTGatewayClient
Added
WebSocketClient&AsyncWebSocketClientto stream requests fromStreamingEndpointforFlow&AsyncFlowrespectivelyFlow
Enabled websocket based clients for REST gateway
Notes & Issues:
With the current implementation, gateway is blocking, as we wait for each response to come back on the websocket(FIXED)needsfail -Flow().add().add(needs='gateway')(TO BE FIXED)bytesorjsonat the gateway & the client. Hence the performance might be poor compared to gRPC.bytesonly for now. Need to add support forjsonsend&recvin websockets when compared togrpc.async_call- websockets-commentwsprotolibrary overwebsocketswith UvicornServer, which adds a new dependency to jina - This PR was never merged in uvicorn.