Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support: yield sse_starlette.ServerSentEvent in /stream #701

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

itok01
Copy link

@itok01 itok01 commented Jul 3, 2024

Allows the /stream endpoint to return sse_starlette.ServerSentEvent from runnable, allowing you to return custom events such as event: error.

The reason for not using langserve.server_sent_events.ServerSentEvent is that is unbable to check with isinstance() whether an object is a langserve.server_sent_events.ServerSentEvent.

Example

Run ./examples/custom_events/server.py

/stream:

$ curl -X POST -d '{"input":{},"config":{}}' 'http://127.0.0.1:8000/stream'
event: metadata
data: {"run_id": "40239ec7-e492-4c26-aac7-4380df1ad050"}

event: data
data: {"message":"Message 0"}

event: data
data: {"message":"Message 1"}

event: data
data: {"message":"Message 2"}

event: data
data: {"message":"Message 3"}

event: error
data: {"message":"An error occurred"}

event: end

/invoke:

$ curl -X POST -d '{"input":{},"config":{}}' 'http://127.0.0.1:8000/invoke'
{"output":{"message":"An error occurred"},"metadata":{"run_id":"06078459-1945-46d0-97ec-d7bec22f408a","feedback_tokens":[]}}

This commit adds support for Server-Sent Events (SSE) in the APIHandler class. It imports the necessary modules from `sse_starlette` and includes a check for the `ServerSentEvent` class. If SSE is supported and the chunk is an instance of `ServerSentEvent`, it yields the chunk. This change improves the functionality of the APIHandler by enabling SSE support.
Copy link

cla-bot bot commented Jul 3, 2024

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the username @itok01 on file. In order for us to review and merge your code, please complete the Individual Contributor License Agreement here https://forms.gle/AQFbtkWRoHXUgipM6 .

This process is done manually on our side, so after signing the form one of the maintainers will add you to the contributors list.

For more details about why we have a CLA and other contribution guidelines please see: https://github.com/langchain-ai/langserve/blob/main/CONTRIBUTING.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant