fix: best-effort aclose() of httpx AsyncClient instead of leaving transports to GC#72
Merged
Merged
Conversation
…nsports to GC Three paths dropped the async client without closing it, orphaning its transports to the cyclic GC whose _SelectorTransport.__del__ socket finalizers then ran on arbitrary threads (false TMPRL1101 deadlocks in Temporal workers — valiot/python-tooling#151): 1. _get_async_client's loop-closed branch 2. async_execute's 'Event loop is closed' retry 3. _close() / __del__ A new _drop_async_client() helper awaits aclose() best-effort (swallowing failures when the original loop is gone) and serves sites 1, 2 and async_cleanup; _close() schedules aclose() on the running loop via call_soon_threadsafe when one exists. Also fixes async_cleanup never actually closing (its get_timeout() probe hit a nonexistent method) and removes the unused _close_async_client. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Overall Assessment
This PR fixes three leak paths for httpx.AsyncClient instances in GraphQLClient. The implementation introduces a _drop_async_client() helper for best-effort aclose() and updates _close() (sync, called by __del__) to schedule aclose() via call_soon_threadsafe when a running loop exists. New hermetic tests cover the three leak sites plus error swallowing. Version and changelog are updated per repo conventions. No blocking bugs found.
Findings
No actionable findings.
Notes
- The broken
get_timeout()probe in_get_async_client(line 1097) is deliberately left untouched per the PR description; client-reuse behavior is unchanged and the leak is now closed regardless. - The broad
except Exceptionin_drop_async_clientand_closeis intentional for best-effort cleanup when the original loop is gone; the docstring explains the rationale. - Test coverage for the lifecycle fix is comprehensive and follows the repo's "hermetic, no sockets, no sleeps" pattern.
palantir-valiot Bot
pushed a commit
that referenced
this pull request
Jun 12, 2026
…nsports to GC (#72) Three paths dropped the async client without closing it, orphaning its transports to the cyclic GC whose _SelectorTransport.__del__ socket finalizers then ran on arbitrary threads (false TMPRL1101 deadlocks in Temporal workers — valiot/python-tooling#151): 1. _get_async_client's loop-closed branch 2. async_execute's 'Event loop is closed' retry 3. _close() / __del__ A new _drop_async_client() helper awaits aclose() best-effort (swallowing failures when the original loop is gone) and serves sites 1, 2 and async_cleanup; _close() schedules aclose() on the running loop via call_soon_threadsafe when one exists. Also fixes async_cleanup never actually closing (its get_timeout() probe hit a nonexistent method) and removes the unused _close_async_client. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
14 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
pygqlc orphaned
httpx.AsyncClientinstances on three paths, leaving their transports to Python's cyclic GC. The orphaned_SelectorTransport.__del__ -> _sock.close()finalizers then ran during GC sweeps on arbitrary threads — prod logs literally showException ignored while calling deallocator <_SelectorTransport.__del__>— which contributed to false TMPRL1101 deadlock detections in Temporal workers. valiot/python-tooling#151 mitigated the worker side withgc.freeze; this PR fixes the leak at its source.The three leak sites
_get_async_client— when it decided the cached client's event loop was closed, it didself._async_client = Nonewithout closing ("Intentionally don't try to close the old client…").async_executeretry — on"Event loop is closed"duringpost, it droppedself._async_client = Nonewithout close and recreated._close()(called by__del__) — sync context; set_async_client = Nonewith no close attempt even when a loop was running.The fix (one helper, three call sites)
_drop_async_client(): swaps the client out and best-effortawait client.aclose(), swallowing (and WARN-logging) failures when the client's original loop is genuinely gone. Used at sites 1 and 2, and byasync_cleanup(which keeps its semantics but loses its duplicated probe logic)._close()(site 3, sync): if this thread has a running loop, scheduleaclose()on it vialoop.call_soon_threadsafe(lambda: loop.create_task(client.aclose()))—__del__can run on any thread. No usable loop → GC fallback, exactly as before.Bonus bug found while testing
The loop-closed "probe" called
await self._async_client.get_timeout()— a method that does not exist onhttpx.AsyncClient. The resultingAttributeErrormatched the"has no attribute"branch, so:_get_async_clientrecreated the client on every call (and leaked the old one every time — this is why the leak was so loud in prod), andasync_cleanupalways took the "let GC handle it" branch and never closed anything.With this PR, recreation now closes the old client first and
async_cleanupactually closes. I deliberately did not touch the probe itself (client-reuse behavior is unchanged: still recreates per call, now leak-free); fixing reuse properly is a separate concern. Also removed the unused_close_async_clientprivate method, superseded by the helper.Type of change
How Has This Been Tested?
New hermetic regression suite
tests/pygqlc/gql_client/test_async_client_lifecycle.py(no real sockets, no sleeps; singleton-bypass fixture pattern fromtest_subscribe.py):_get_async_clientacloses the stale client when replacing it (loop-closed branch simulated)async_execute's "Event loop is closed" retry acloses the stale client before retrying_close()under a running loop schedulesaclose()(real asyncio loop); without a loop it falls back silently_drop_async_clientswallowsaclose()exceptions and still drops the clientasync_cleanupactually closes a live clientRed-first verified: 4/6 fail against pre-fix code (the other 2 guard preserved fallback behavior). Full suite: 37 passed locally (live-API tests skip-error without env secrets, as usual).
uvx ruff format --checkclean; bandit (-s B101 -ll) exit 0;uv lock --upgraderun anduv.lockcommitted per check-outdated policy.Checklist:
pygqlc/__version__.py)🤖 Generated with Claude Code