Skip to content

OpenAI: chat.completions.stream() helper method not instrumented #114

@braintrust-bot

Description

@braintrust-bot

Gap

The OpenAI Python SDK provides a client.chat.completions.stream() helper method that is distinct from client.chat.completions.create(stream=True). This helper is not instrumented — calling it on a wrapped client produces no Braintrust span.

CompletionsV1Wrapper (line 776 of oai.py) only defines create(). When .stream() is called, it falls through via NamedWrapper.__getattr__ (line 24) to the underlying unwrapped Completions resource. The request succeeds but is invisible to tracing.

How .stream() differs from .create(stream=True)

The .stream() helper (documented in the openai-python helpers.md):

  • Returns a context manager (ChatCompletionStream)
  • Provides an event-based API with structured event objects instead of raw chunks
  • Automatically accumulates deltas across streaming chunks
  • Offers convenience methods like .get_final_completion() and .until_done()
  • Is the SDK-recommended pattern for streaming in many OpenAI guides
# Pattern 1 — instrumented today
stream = client.chat.completions.create(stream=True, ...)
for chunk in stream:
    ...

# Pattern 2 — NOT instrumented
with client.chat.completions.stream(...) as stream:
    for event in stream:
        ...
    final = stream.get_final_completion()

The same gap likely applies to AsyncCompletionsV1Wrapper (line 803) for the async variant.

What is missing

  • client.chat.completions.stream() (sync) — needs a stream() method on CompletionsV1Wrapper
  • client.chat.completions.stream() (async) — needs a stream() method on AsyncCompletionsV1Wrapper

Braintrust docs status

unclear — the OpenAI integration page mentions streaming support but does not distinguish between .create(stream=True) and .stream().

Upstream sources

Local files inspected

  • py/src/braintrust/oai.py:
    • CompletionsV1Wrapper (line 776) — only defines create()
    • AsyncCompletionsV1Wrapper (line 803) — only defines async create()
    • NamedWrapper.__getattr__ (line 24) — delegates unknown attrs to the unwrapped object
  • py/src/braintrust/wrappers/test_openai.py — no tests for chat.completions.stream(), all streaming tests use create(stream=True)
  • py/noxfile.py — tests openai versions latest, 1.77.0, 1.71, 1.91, 1.92; .stream() helper exists in all of these

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions