Skip to content

test(openai): add vcr regression coverage for stream helper#214

Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit intomainfrom
abhi-test-114-openai-stream-helper
Apr 7, 2026
Merged

test(openai): add vcr regression coverage for stream helper#214
Abhijeet Prasad (AbhiPrasad) merged 1 commit intomainfrom
abhi-test-114-openai-stream-helper

Conversation

@AbhiPrasad
Copy link
Copy Markdown
Member

@AbhiPrasad Abhijeet Prasad (AbhiPrasad) commented Apr 7, 2026

AI Summary

Add VCR regression coverage for OpenAI stream helper APIs.

This PR extends the existing chat.completions.stream() coverage to also test responses.stream() in both sync and async flows. The new tests verify that:

  • unwrapped clients preserve normal SDK behavior and emit no Braintrust spans
  • wrapped clients preserve the helper interface and do emit Braintrust spans
  • streamed output is still consumable through the helper APIs
  • final accumulated responses remain accessible after streaming completes

This covers regression cases related to:

What changed

Added tests

  • sync + async VCR tests for responses.stream()
  • assertions that helper APIs still work end-to-end:
    • streamed events can be consumed
    • get_final_response() still works
    • tracing spans are emitted only for wrapped clients

Added cassettes

  • test_openai_responses_stream_helper.yaml
  • test_openai_responses_stream_helper_async.yaml

Why

responses.stream() is a first-class OpenAI SDK helper and should stay covered by regression tests the same way chat.completions.stream() is. Current instrumentation works through the existing wrapped streaming path, but there was no explicit VCR coverage protecting that behavior.

Test strategy

Used a red/green approach:

  • Red: temporarily removed the relevant wrapping path and confirmed the new stream-helper tests failed with zero spans emitted
  • Green: restored the current implementation and confirmed the tests passed

Validation

Targeted test run:

cd py && .nox/test_openai-latest/bin/pytest -p no:braintrust \
  src/braintrust/integrations/openai/test_openai.py \
  -k "responses_stream_helper or openai_chat_streaming_sync or openai_chat_streaming_async"

Passed:

  • test_openai_responses_stream_helper
  • test_openai_responses_stream_helper_async
  • existing chat stream helper coverage

@starfolkai
Copy link
Copy Markdown

starfolkai bot commented Apr 7, 2026

Drop into this review session: sfk devbox connect pr-214-braintrust-sdk-python-a8ca --attach

No serious bugs or issues found 🎉

@AbhiPrasad Abhijeet Prasad (AbhiPrasad) force-pushed the abhi-test-114-openai-stream-helper branch from eda75fc to 2d5e3d9 Compare April 7, 2026 16:32
Add explicit sync and async VCR tests for both chat.completions.stream() and responses.stream() and verify the helper paths produce Braintrust spans when the client is wrapped.\n\nAlso keep coverage for the OpenAIIntegration.setup() path so global instrumentation continues to trace the stream helpers and preserve helper-specific behavior.\n\nThese tests cover the regressions discussed in #114 and #170.
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) force-pushed the abhi-test-114-openai-stream-helper branch from 2d5e3d9 to 2a2505d Compare April 7, 2026 16:55
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) merged commit b8eaf0e into main Apr 7, 2026
29 checks passed
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) deleted the abhi-test-114-openai-stream-helper branch April 7, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants