Skip to content

Lease extend#398

Merged
v1r3n merged 4 commits intomainfrom
lease_extend
Apr 19, 2026
Merged

Lease extend#398
v1r3n merged 4 commits intomainfrom
lease_extend

Conversation

@v1r3n
Copy link
Copy Markdown
Contributor

@v1r3n v1r3n commented Apr 18, 2026

Summary

Lease Extension

Workers can now opt into automatic lease extension so long-running tasks don't time out:

  • New LeaseExtendWorker base class and lease_extend_enabled parameter on @worker_task
  • Background thread periodically calls extendLease before the current lease expires
  • Thread-safe implementation with proper cleanup on task completion
  • Configurable via worker properties

Memory Leak Fix (Fixes #395)

Long-running worker processes experienced ~450 KB/min memory growth per subprocess due to three issues in the HTTP layer:

  1. httpx.Response <-> BoundSyncStream reference cycleRESTResponse now eagerly reads resp.text into self.data and breaks the cycle by nulling resp.stream / resp._request
  2. RESTResponse(io.IOBase) retained full httpx.Response — dropped io.IOBase inheritance, removing __del__ finalizer overhead
  3. self.last_response write-only retention — removed dead write that pinned the last response for the lifetime of ApiClient

Changes applied to both sync (rest.py, api_client.py) and async (async_rest.py, async_api_client.py) codepaths. Also fixed silently broken AuthorizationException test mocks that were using the removed .resp attribute.

Test plan

  • Memory leak regression test (test_memory_leak.py) — weakref-based test proving httpx.Response objects are GC-able after RESTResponse construction
  • Standalone reproduction script (repro_memory_leak.py) — tracemalloc validation showing 11.7 KB growth over 2000 requests (vs ~450 KB/min before fix)
  • All 1710 unit/backward-compat/serialization tests pass
  • Fixed AuthorizationException test mocks to use .data instead of dead .resp attribute

🤖 Generated with Claude Code

v1r3n and others added 3 commits April 18, 2026 14:19
- Fix lease leak: when __execute_task returns None (ASYNC_TASK_RUNNING),
  keep the lease tracked instead of untracking in the finally block.
  Untrack in __check_completed_async_tasks when the async task finishes.
- Remove dead isinstance(task_result, TaskInProgress) checks — __execute_task
  always wraps TaskInProgress into TaskResult before returning.
- Add threading.Lock around _lease_info dict for PEP 703 free-threaded
  Python compatibility (safe beyond CPython GIL).
- Log exception cause in _send_heartbeat retry failures (was swallowed).
- Extract LeaseInfo dataclass and constants into shared lease_tracker.py
  module to eliminate duplication between task_runner and async_task_runner.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…395)

RESTResponse now eagerly reads resp.text into self.data and breaks the
httpx Response <-> BoundSyncStream cycle by nulling resp.stream and
resp._request. Drops io.IOBase inheritance (removes __del__ finalizer
overhead). Removes write-only self.last_response retention. Adds
json(), getheader() convenience methods to RESTResponse.

Changes applied to both sync (rest.py, api_client.py) and async
(async_rest.py, async_api_client.py) codepaths.

Fixes #395

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@nohup23
Copy link
Copy Markdown

nohup23 commented Apr 18, 2026

..

@nohup23 nohup23 self-requested a review April 18, 2026 22:32
- Wrap resp._request = None in try/except AttributeError for forward
  compat with future httpx versions
- Fix async_rest.py comment: BoundSyncStream → BoundAsyncStream
- Fix test_task_runner_coverage.py mocks: use mock_http_resp.data
  instead of dead mock_http_resp.resp (AuthorizationException now
  reads .data, not .resp.text)

Follow-up to #395

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@v1r3n v1r3n merged commit 387d6db into main Apr 19, 2026
1 check passed
@v1r3n v1r3n deleted the lease_extend branch April 19, 2026 03:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory leak: httpx.Response objects never garbage collected due to reference cycle in BoundSyncStream

2 participants