tests: add local SSE harness (opt-in) and transport timeout coverage #2655
Conversation
… client-timeout wrapper test for now
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
Codex Review: Here are some suggestions.
About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".
| import pytest | ||
| import httpx | ||
| from openai import AsyncOpenAI | ||
|
|
||
| @pytest.mark.anyio | ||
| async def test_openai_client_bubbles_readtimeout_from_transport(): | ||
| async def handler(request: httpx.Request) -> httpx.Response: | ||
| # Força um ReadTimeout vindo do transporte | ||
| raise httpx.ReadTimeout("forced timeout", request=request) | ||
|
|
||
| transport = httpx.MockTransport(handler) | ||
| http_client = httpx.AsyncClient( | ||
| transport=transport, | ||
| timeout=httpx.Timeout(0.05, connect=0.05, read=0.05, write=0.05), | ||
| base_url="https://api.openai.test", | ||
| ) | ||
| client = AsyncOpenAI( | ||
| api_key="dummy", | ||
| http_client=http_client, | ||
| base_url="https://api.openai.test" | ||
| ) | ||
|
|
||
| with pytest.raises(httpx.ReadTimeout): | ||
| # Qualquer chamada simples; o MockTransport intercepta |
There was a problem hiding this comment.
[P1] Expect APITimeoutError rather than raw httpx.ReadTimeout
The new timeout test asserts that AsyncOpenAI bubbles the underlying httpx.ReadTimeout unchanged. However the client intentionally wraps any httpx.TimeoutException into openai.APITimeoutError before re‑raising (src/openai/_base_client.py, both sync and async paths). Calling await client.models.list() with a transport that raises httpx.ReadTimeout will therefore raise APITimeoutError, causing this test to fail immediately. Either the assertion should be updated to expect APITimeoutError, or the client code would need to be changed to stop wrapping timeouts, which contradicts current behavior.
Useful? React with 👍 / 👎.
|
Hello OpenAI team! 👋
The harness is opt-in only (via UNXFAIL_SSE=1) to avoid breaking CI by default. I’d really appreciate your feedback on whether I should refine this further or leave it scoped as-is. Thanks! 🙏 |
|
Thanks for the PR but the unified changes here are not something we'd want to expose, and we already have other tests for SSE. |
|
Hi Robert, thanks for the feedback and for taking the time to review this PR. I understand the reasoning and will focus on other areas where I can contribute. I also have some other open PRs in the repo that haven’t been reviewed yet — please let me know if I should make any adjustments there to help the process. |
2 participants
This PR introduces new test utilities and coverage for the OpenAI Python SDK.
✅ What we added / improved
Local SSE harness (tests/utils/sse_server.py)
Minimal aiohttp server that emits a well-formed SSE event sequence.
Opt-in via UNXFAIL_SSE=1 to avoid breaking CI by default.
Supports both “unified” and “legacy” streaming flows.
New tests
tests/test_streaming_with_local_sse.py: validates SSE event ordering and stream lifecycle (skipped unless UNXFAIL_SSE=1).
tests/test_timeouts_client.py: ensures httpx.ReadTimeout is correctly surfaced as openai.APITimeoutError.
Adjusted tests/conftest.py and touched test_streaming_unified_integration.py / test_timeouts.py to align with the new harness.
SSE tests: still flaky when enabled; state machine handling of content_index vs index is unresolved.
Audio tests: several transcription/translation tests are failing (tests/api_resources/audio/test_transcriptions.py, test_translations.py). These failures are pre-existing and unrelated to the changes in this PR.
🎯 Next steps
Revisit SSE event schema to align exactly with the SDK’s ResponsesState accumulator.
Investigate the failing audio tests separately (likely model / fixture mismatch).
This PR is intentionally scoped to infrastructure and harness work — not fixing all SSE edge cases or audio failures.
It gives us a baseline for timeouts (solid ✅) and a controlled SSE harness we can iterate on later.