Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Config entry tasks are being leaked #96980

Closed
bdraco opened this issue Jul 21, 2023 · 4 comments · Fixed by #96981
Closed

Config entry tasks are being leaked #96980

bdraco opened this issue Jul 21, 2023 · 4 comments · Fixed by #96981
Assignees
@bdraco
Labels
memory-leak
Milestone
2023.7.3

Comments

@bdraco
Copy link
Member

bdraco commented Jul 21, 2023

The problem

_async_process_on_unload is showing 1000s of tasks being waited for on

unload_leaking_tasks

What version of Home Assistant Core has the issue?

2023.7.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response
@bdraco bdraco self-assigned this Jul 21, 2023
@bdraco
Copy link
Member Author

bdraco commented Jul 21, 2023 

2023-07-20 14:03:49.888 WARNING (MainThread) [homeassistant.config_entries] Config entry a4f536357a4867d2bfdeb656b4c8b5e1 with domain emonitor is being unloaded, waiting for tasks: {<Task finished name='Task-7840' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12990' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12606' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-11867' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12323' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task pending name='Task-13725' coro=<DataUpdateCoordinator.async_shutdown() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> cb=[set.remove()]>}, background tasks: set()
2023-07-20 14:03:50.272 WARNING (MainThread) [homeassistant.config_entries] Config entry 15f2f15cc9084452ab526ea1181483eb with domain ipp is being unloaded, waiting for tasks: {<Task finished name='Task-12992' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12326' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-11866' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12608' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task pending name='Task-13729' coro=<DataUpdateCoordinator.async_shutdown() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> cb=[set.remove()]>, <Task finished name='Task-8546' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>}, background tasks: set()
2023-07-20 14:04:04.236 WARNING (MainThread) [homeassistant.config_entries] Config entry 25c21063365daf23bbb99667ab5b4387 with domain emonitor is being unloaded, waiting for tasks: {<Task finished name='Task-12706' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-13291' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-9481' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task pending name='Task-13834' coro=<DataUpdateCoordinator.async_shutdown() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> cb=[set.remove()]>}, background tasks: set()
2023-07-20 14:04:54.803 WARNING (MainThread) [homeassistant.config_entries] Config entry 17c56fff3c731436a1e9d07ce31c3e59 with domain starlink is being unloaded, waiting for tasks: {<Task finished name='Task-12538' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-13542' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-4081' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task pending name='Task-14340' coro=<DataUpdateCoordinator.async_shutdown() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> cb=[set.remove()]>, <Task finished name='Task-11218' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12902' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-11934' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>}, background tasks: set()

Every retry is leaking another DataUpdateCoordinator.async_shutdown

@bdraco
Copy link
Member Author

bdraco commented Jul 21, 2023

Its because on unload we add them but don't add the callback to remove them

        if self._on_unload is not None:
            while self._on_unload:
                if job := self._on_unload.pop()():
                    self._tasks.add(hass.async_create_task(job))

@bdraco bdraco added this to the 2023.7.3 milestone Jul 21, 2023
@bdraco bdraco mentioned this issue Jul 21, 2023
Merged
@bdraco
Copy link
Member Author

bdraco commented Jul 21, 2023

regressed in #91531

@bdraco bdraco mentioned this issue Jul 21, 2023
Merged
20 tasks
@bdraco
Copy link
Member Author

bdraco commented Jul 21, 2023 •
edited

after 15m its already leaking quite a bit

name='Task-15465' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-16166' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-17570' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-18253' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-16844' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-7840' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12990' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12606' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-11867' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12323' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task pending name='Task-18927' coro=<DataUpdateCoordinator.async_shutdown() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> cb=[set.remove()]>, <Task finished name='Task-13725' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-14621' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>}, background tasks: set()
2023-07-20 14:13:34.016 WARNING (MainThread) [homeassistant.config_entries] Config entry 15f2f15cc9084452ab526ea1181483eb with domain ipp is being unloaded, waiting for tasks: {<Task finished name='Task-12992' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-15473' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12326' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-14631' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-11866' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12608' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-17579' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-13729' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-16168' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-18260' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task pending name='Task-18932' coro=<DataUpdateCoordinator.async_shutdown() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> cb=[set.remove()]>, <Task finished name='Task-8546' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-16845' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>}, background tasks: set()
2023-07-20 14:14:17.058 WARNING (MainThread) [homeassistant.config_entries] Config entry 17c56fff3c731436a1e9d07ce31c3e59 with domain starlink is being unloaded, waiting for tasks: {<Task finished name='Task-12538' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-13542' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-15913' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-17956' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-4081' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-16562' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task pending name='Task-19214' coro=<DataUpdateCoordinator.async_shutdown() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> cb=[set.remove()]>, <Task finished name='Task-14340' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-15141' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-11218' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-12902' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-18576' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-17274' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>, <Task finished name='Task-11934' coro=<DataUpdateCoordinator.async_shutdown() done, defined at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:166> result=None>}, background tasks: set()
2023-07-20 14:14:17.341 ERROR (SyncWorker_3) [roombapy.remote_client] Can't connect to 192.168.107.20, error: [Errno 111] Connection refused

surprised I'm the first to notice the root cause

@github-actions github-actions bot locked and limited conversation to collaborators Aug 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@bdraco bdraco

Labels
memory-leak
Projects
None yet
Milestone
2023.7.3
Development

Successfully merging a pull request may close this issue.

Fix task leak on config entry unload/retry home-assistant/core
1 participant
@bdraco