add metadata event to /stream for collecting run_id of individual runs #180
Conversation
jakerachleff
changed the title
| @@ -681,10 +681,23 @@ async def _stream() -> AsyncIterator[dict]: | |||
| ) from validation_exception | |||
|
|
|||
| try: | |||
| config_w_callbacks = config.copy() | |||
| event_aggregator = AsyncEventAggregatorCallback() | |||
| config_w_callbacks["callbacks"] = [event_aggregator] | |||
There was a problem hiding this comment.
ooc: should we be merging rather than overwriting these?
There was a problem hiding this comment.
I'll defer to @eyurtsev since I'm copying his pattern here
There was a problem hiding this comment.
Historically was fine to overwrite. With addition of per request modifier, we might want to merge if a user is adding custom callbacks
There was a problem hiding this comment.
I'd say keep as is for now and we can follow up with a separate PR to merge
There was a problem hiding this comment.
works for me, will merge
langserve/server.py
Outdated
| if not has_sent_metadata and event_aggregator.callback_events: | ||
| yield { | ||
| "event": "metadata", | ||
| "data": { |
There was a problem hiding this comment.
Do we not need to serialize the data into json? I think it doesn't do this by default for streaming
There was a problem hiding this comment.
ah yeah that's a good callout - that makes sense why the output had single quotes instead of double quotes! I'll make this change before pushing
There was a problem hiding this comment.
Before:
After:
Adds a new event with type
metadatathat can return metadata about a run. This can be used for getting a run_id from the/streamAPITesting
test_server_clientexamples/llm/server.pylocally and ensured the returned run_id from the metadata event matched the one logged to langsmith