Skip to content

[Service Bus] aio AutoLockRenewer: prune completed futures to fix memory leak - #48403

Draft
EldertGrootenboer wants to merge 2 commits into
mainfrom
fix/aio-autolockrenewer-futures-leak-48366
Draft

[Service Bus] aio AutoLockRenewer: prune completed futures to fix memory leak#48403
EldertGrootenboer wants to merge 2 commits into
mainfrom
fix/aio-autolockrenewer-futures-leak-48366

Conversation

@EldertGrootenboer

Copy link
Copy Markdown
Member

Summary

Fixes #48366.

The async AutoLockRenewer appended each renewal asyncio.Future to an internal list _futures in register() and never removed it on completion — the list was drained only by close(). A long-lived renewer (the documented one-renewer-per-app pattern) therefore accumulated one future per registered message for its whole lifetime: memory grew proportional to the total messages processed, reclaimed only on close().

Fix

  • _futures is now a set, and register() registers renew_future.add_done_callback(self._futures.discard) so each future is removed the moment its renewal coroutine completes (success, timeout, or error). The collection stays bounded by the number of active renewals rather than the all-time count of register() calls.
  • close() is unchanged (await asyncio.wait(self._futures) still awaits any remaining active futures; asyncio.wait snapshots its input, so callback-driven removal during the await is safe).
  • No public API change — private state, a done-callback, and a typing import.

Tests

New tests/unittests/test_auto_lock_renewer_futures.py (offline, no network):

  • bounded-after-settle: registered renewals complete and leave the set empty (fails against the unfixed code)
  • close() still awaits an active renewal
  • an errored renewal is still pruned
  • over-prune guard: active futures are retained while completed ones are removed

The full offline unit suite passes. The Python sync renewer is unaffected (it self-drains via a queue.Queue), and the .NET/Java/JS SDKs prune renewal handles on completion, so this is an aio-only fix.

The async AutoLockRenewer appended each renewal future to an internal
list and only drained it on close(), so a long-lived renewer (the
documented one-per-app pattern) accumulated one future per registered
message for its whole lifetime - memory growth proportional to the total
messages processed, reclaimed only on close().

Store the futures in a set and remove each one on completion via
renew_future.add_done_callback(self._futures.discard), keeping the
collection bounded by the number of active renewals. close() is
unchanged. Adds unit tests covering bounded-after-settle, close-awaits-
active, errored-renewal-pruned, and the over-prune guard.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Ready to approve

The implementation safely bounds retained futures and includes focused coverage for completion and shutdown behavior.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Fixes #48366 by pruning completed async lock-renewal futures, preventing unbounded memory growth.

Changes:

  • Tracks active futures in a set and removes them on completion.
  • Adds offline regression tests for completion, errors, and shutdown.
  • Documents the fix in the changelog.
File summaries
File Description
_async_auto_lock_renewer.py Prunes completed renewal futures.
test_auto_lock_renewer_futures.py Adds regression coverage.
CHANGELOG.md Records the memory-leak fix.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ServiceBus] aio AutoLockRenewer never prunes completed futures from _futures, leaking memory over a long-running consumer

2 participants