Skip to content

fix: fix streaming in java sdk#95

Merged
ericdong-langchain merged 9 commits intomainfrom
ericdong/lsdk-132-fix-java-sdk-streaming
Mar 9, 2026
Merged

fix: fix streaming in java sdk#95
ericdong-langchain merged 9 commits intomainfrom
ericdong/lsdk-132-fix-java-sdk-streaming

Conversation

@ericdong-langchain
Copy link
Copy Markdown
Collaborator

@ericdong-langchain ericdong-langchain commented Mar 5, 2026

Issues

  • Chat streaming: The span ends when createStreaming() returned, so the trace was closed before the stream finished. Completion text, tool calls, and token usage were not on the span, and gen_ai.input.messages could be wrong or empty.

  • Completion format: Non-streaming set gen_ai.completion to raw assistant text aligned this with streaming.

  • Responses API streaming: Span ended on createStreaming return instead of stream close.

  • OTel integration: No way to attach LangSmith to an existing SdkTracerProvider.

Fix

  • Chat streaming: End the span in TracedChatStreamResponse.close() so it closes when the stream is closed. Use ChatStreamAggregator to aggregate content, tool calls, and usage from chunks and apply them in close(). Always set gen_ai.input.messages from formatInputMessages(params), set gen_ai.prompt only when user text is found. Request stream_options.include_usage=true so the final chunk includes usage.

  • Completion format: For both streaming and non-streaming, set gen_ai.completion to a JSON string {"messages":[...]}, with streaming building the assistant message from the aggregator.

  • Responses API streaming: Wrap the stream in TracedResponseStreamResponse in close(), use ResponseAccumulator.response() to read usage and status and set them on the span, then end the span.

  • OTel config: Add buildSpanProcessor() that returns a SpanProcessor so callers can add it to an existing SdkTracerProvider. Share setup with build() via a private createSpanProcessor() helper.

  • Also added a streaming example: StreamingLangSmithExample; run with:
    ./gradlew :langsmith-java-example:run -Pexample=StreamingLangSmith

@ericdong-langchain ericdong-langchain marked this pull request as ready for review March 5, 2026 19:11
@ericdong-langchain ericdong-langchain merged commit 76da93f into main Mar 9, 2026
12 checks passed
@ericdong-langchain ericdong-langchain deleted the ericdong/lsdk-132-fix-java-sdk-streaming branch March 9, 2026 20:07
@stainless-app stainless-app bot mentioned this pull request Mar 9, 2026
@stainless-app stainless-app bot mentioned this pull request Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants