feat(node): Add streaming to orchestrion OpenAI integration#21886
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| const instrumented = instrumentStream({ [Symbol.asyncIterator]: iterate }, span, recordOutputs ?? false); | ||
| (result as { [Symbol.asyncIterator]: () => AsyncIterator<unknown> })[Symbol.asyncIterator] = () => instrumented; | ||
|
|
||
| return true; |
There was a problem hiding this comment.
Open streaming spans never end
Medium Severity
When wrapStreamResult defers span completion for stream: true, the span is only finished inside instrumentStream's finally block after async iteration. If the caller resolves create() but never drives the returned Stream (no for await, abandoned reference, or fire-and-forget), deferSpanEnd skips the normal span.end() path and the gen_ai span can remain open indefinitely.
Reviewed by Cursor Bugbot for commit 115b544. Configure here.
nicohrubec
force-pushed
the
nh/openai-orchestrion-streaming
branch
2 times, most recently
from
58d2833 to
345dcba
Compare
size-limit report 📦
|
Instruments streaming `chat.completions.create` / `responses.create` by patching the returned `Stream`'s async iterator in place (via `bindTracingChannelToSpan`'s `deferSpanEnd`) so the reused `instrumentStream` accumulates chunk state and ends the span when iteration completes. Exports `instrumentStream` from core. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nicohrubec
force-pushed
the
nh/openai-orchestrion-streaming
branch
from
345dcba to
9ca6a54
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9ca6a54. Configure here.
| const instrumented = instrumentStream({ [Symbol.asyncIterator]: iterate }, span, recordOutputs ?? false); | ||
| result[Symbol.asyncIterator] = () => instrumented; | ||
|
|
||
| return true; |
There was a problem hiding this comment.
Stream errors leave span ok
Medium Severity
When deferSpanEnd hands off a streaming result, wrapStreamResult never uses the provided end callback. If async iteration fails (network abort, parse error, etc.), instrumentStream still runs endStreamSpan in its finally block and ends the span without error status, and the integration’s captureError / error annotation path is skipped—unlike rejected create() calls or the mysql orchestrion deferSpanEnd pattern.
Reviewed by Cursor Bugbot for commit 9ca6a54. Configure here.
|
Folded directly into #21877 (fast-forwarded onto its base) — decided to ship the full OpenAI orchestrion surface in one PR. |
1 participant
Stacked on #21877 (base:
nh/openai-orchestrion) — review/merge that first.Adds streaming (
stream: true) support forchat.completions.createandresponses.createin the orchestrion OpenAI integration.A streaming
create()resolves to aStreamthe caller consumes later, so the span must outlive the channel event and be enriched from the accumulated chunks. We can't swap whatcreatereturns, but theStreaminctx.resultis the same instance the caller gets andasyncEndfires before the caller iterates — so we patch its async iterator in place to run through the reusedinstrumentStream(which accumulates tokens/finish-reasons/text and ends the span when iteration completes), viabindTracingChannelToSpan'sdeferSpanEnd. Non-streaming and errored results keep the existing path.Changes:
packages/core: exportinstrumentStream(pure generator, browser-safe).integrations/tracing-channel/openai.ts: drop the streaming opt-out; adddeferSpanEndthat patches the stream instance and hands span-ending toinstrumentStream.🤖 Generated with Claude Code