feat(python-sdk): move the volume content client onto pyqwest#1602
feat(python-sdk): move the volume content client onto pyqwest#1602mishushakov wants to merge 3 commits into
Conversation
Volume/AsyncVolume file operations join the API client on the ApiPyqwestTransport + connection-retry stack; transport caches become process-global keyed by proxy. httpx.Proxy values reduce to plain proxy URLs; stream_idle_timeout is accepted but a no-op at this point (bounded by the transport in the next commits). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n a dedicated transport reqwest's read timer keeps ticking while a request body is sent and while waiting for the response head, so a shared read_timeout would kill slow uploads; streamed reads get their own transport with read_timeout=60s (JS SDK parity), uploads and unary calls stay unbounded-per-read. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ead_file asyncio.wait_for around each read (response head and every chunk); explicit values run on the regular transport so values above the 60s transport bound aren't capped and 0 disables idle bounding. Sync keeps the parameter but ignores it — it cannot interrupt a blocking read into the Rust transport. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 8c92831 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
PR SummaryMedium Risk Overview Streamed Reviewed by Cursor Bugbot for commit 8c92831. Bugbot is set up for automated code reviews on this repo. Configure here. |
Package ArtifactsBuilt from c397d0f. Download artifacts from this workflow run. JS SDK ( npm install ./e2b-2.35.4-migrate-volume-to-pyqwest.0.tgzCLI ( npm install ./e2b-cli-2.15.1-migrate-volume-to-pyqwest.0.tgzPython SDK ( pip install ./e2b-2.34.0+migrate.volume.to.pyqwest-py3-none-any.whl |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8c92831. Configure here.
| break | ||
| yield chunk | ||
| finally: | ||
| await stream_cm.__aexit__(None, None, None) |
There was a problem hiding this comment.
Stream enter timeout skips cleanup
Medium Severity
When an explicit stream_idle_timeout is set, asyncio.wait_for wraps stream_cm.__aenter__(). On timeout it cancels that enter before the inner try/finally, so __aexit__ never runs and the httpx stream generator is left suspended. That can leak a connection from the process-global pool on slow response heads.
Reviewed by Cursor Bugbot for commit 8c92831. Configure here.
|
from other PR:
|


What
Stacked on #1601. Migrates the volume content client (
Volume/AsyncVolumefile operations) onto pyqwest via its httpx-compatible transport adapter — the sameApiPyqwestTransport+ connection-retry stack the REST API client uses after #1601.Originally deferred from #1601 because
Volume.read_file(format="stream")relied on httpx's per-readreadtimeout as an idle timeout, which the adapter can't express per request (it converts the httpx timeout dict into a whole-request deadline, and the sync adapter doesn't bound body reads at all). Unblocked by pyqwest's transport-constructorread_timeout, which maps to reqwest'sClientBuilder::read_timeout— verified behaviorally (local slow-chunk server, sync + async) to be a true per-read idle timeout: it resets after each successful read, covers body reads, and a healthy stream longer than the timeout completes untouched.How
e2b/volume/client_sync/__init__.py/client_async/__init__.pymove to the sameApiPyqwestTransport+ connection-retry stack as the API client. Caches become process-global, keyed by (proxy, streaming) — previously one pool per thread (sync) / per event loop (async).read_timeout=60s. It can't live on the shared transport: reqwest's read timer keeps running while a request body is sent and while waiting for the response head (verified empirically — a 2.4 s upload against a 0.5 sread_timeoutdies mid-send), so a sharedread_timeoutwould cut offwrite_fileuploads and slow unary responses longer than the idle bound. Uploads and unary calls stay on a transport without it, bounded by their whole-request deadlines as before.requestTimeoutMs(60 s default) and idle gaps bystreamIdleTimeoutMs ?? requestTimeoutMs; the Python streaming transport'sread_timeoutbounds the response head and each idle gap at 60 s, resetting on every chunk, wire-only (a slow consumer doesn't trip it — verified).AsyncVolume.read_filekeeps honoring an explicitstream_idle_timeoutper call, the same way JS honorsstreamIdleTimeoutMsand feat(python-sdk): migrate envd RPC to the official connectrpc client #1558 bounds stream setup:asyncio.wait_foraround each read (response head and every chunk). Explicit values run on the regular transport, so a value above the 60 s transport bound isn't capped by it and0disables idle bounding entirely, restoring the previous contract. The sync client keeps the parameter but ignores it — it has no way to interrupt a blocking read into the Rust transport, so its bound must live in the transport.request_timeoutbecomes the total-transfer deadline.TimeoutError; the stream wrappers remap it tohttpx.ReadTimeout, keeping the established contract.str,httpx.URL, and reduciblehttpx.Proxyvalues work; inexpressible extras raiseInvalidArgumentException.Testing
tests/test_volume_client.pyrewritten: process-global transport caching (shared across threads and event loops), streaming vs regular transport separation, plus end-to-end streamed reads throughVolume.read_file/AsyncVolume.read_fileagainst a local chunked server — a healthy stream longer than the idle timeout completes (proves the timeout resets per read), a mid-body stall raiseshttpx.ReadTimeout, a slow response head on a non-streamed read is not cut off by the idle bound, and a slow response head on a streamed read is (JS handshake-timeout parity). Asyncstream_idle_timeout: an explicit value aborts a stall, a value above the transport bound isn't capped by it, and0disables idle bounding.403: use of volumes is not enabled); the mock-transport volume content tests and the local-server stream tests cover that path.ruff), typecheck (ty), unit suite: green.Usage example
No API changes for the common path:
🤖 Generated with Claude Code