Skip to content

Conversation

@pawl
Copy link
Contributor

@pawl pawl commented Dec 23, 2021

This is related to my comment on the celery memory leak bug: celery/celery#4843 (comment)

I added some logging to on_poll_start in the redis Transport's register_with_event_loop, and I noticed the event loop continues to poll on redis Transports that were disconnected. This causes some leftover references to the initialized Transport in the event loop's on_tick and prevents the initialized Transport from being garbage collected.

This is similar to the issue @michael-lazar noticed in bug #1 in his comment about the py-amqp memory leak. However, the code for the redis transport is very different because it uses MultiChannelPoller.

This PR fixes the issue by removing the disconnected redis Transport's polling function from the event loop's on_tick when disconnection occurs (to stop the polling in the event loop on the dead Transport).

I tested the fix with this minimal example and was able to confirm it stopped the relevant memory leak.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants