Yaml reload causes memory % increase which is not released #112811

Mariusthvdb · 2024-03-09T10:00:07Z

The problem

using C t load the quick menu and reload Themes consistently ups the memory % with .2 to .5 %, whihc is not released afterwards.

A single reload is not very therefor not that noticeable, but testing themes and doing it often in a row caused this to be a significant rise.

What's more, reloading the complete Yaml (many individual components are no longer available in the Yaml reload page, so user clicks Reload all) causes the system to hang for a very long time.

could it be that

it looks like something still holds on to the old config when you reload

What version of Home Assistant Core has the issue?

2024.4.0.dev20240309

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

core

Link to integration documentation on our website

https://www.home-assistant.io/integrations/homeassistant/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

no errors in log

Additional information

direct results of reloading Theme via 'C Themes'

home-assistant · 2024-03-09T10:00:11Z

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (homeassistant) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of homeassistant can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign homeassistant Removes the current integration label and assignees on the issue, add the integration domain after the command.
@home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
@home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

homeassistant documentation
homeassistant source
_{^{(message by IssueLinks)}}

schellevis · 2024-03-17T14:40:00Z

I experience the same issue on Home Assistant Green (2024.3)

(Except the memory usage spike near 12:00, caused by ESPhome flashing)

erkr · 2024-03-17T19:37:50Z

Since I updated from 2024.2.4 to 2024.3.1 I have a memory loss as well, which seems not related to specific actions:

Update after March the 15th. First time it raised quickly, I rebooted and since then it increases slowly

Mariusthvdb · 2024-03-17T20:18:22Z

my issue really is on the reloading Yaml (themes) especially.

generic memory increase might better be served with another issue? and some logs if you have them.

erkr · 2024-03-17T21:58:50Z

I have nothing in the log's, but in my case I was not making changes that require yaml reloads.
I made a separate issue: #113708

schellevis · 2024-03-18T11:48:21Z

To clarify: my issue is in fact related to yaml reloading.

Ftown14 · 2024-03-19T11:19:35Z

I am seeing this issue as well. I am normally at ~35% usage, but last few days I have been doing lots of tweaks and reloading my YAML a lot. This has led to my memory usage increasing exponentially and rebooting is the only fix.

joostlek · 2024-03-19T12:02:14Z

Right, let me send the copy pasta related to performance issues:

Include a callgrind file from the https://www.home-assistant.io/integrations/profiler/ profiler.start service
Include a py-spy from https://community.home-assistant.io/t/instructions-to-install-py-spy-on-haos/480473. (be sure to zip them up before posting as github mutates svg files)
If memory is leaking, include 4 hours of RAW logs (Settings -> System -> Logs -> Download) from the https://www.home-assistant.io/integrations/profiler/ profiler.start_log_objects service with the default settings

Maybe because because of the reload the memory can rise quicker, we don't need full 4 hours, but let's see what comes out of it

related issue #112811 When the exception hits, the config will end up being saved in the traceback so the memory is never released. This matches the check_config code to homeassistant.config to avoid having the exception thrown.

home-assistant · 2024-04-01T22:31:46Z

system_log documentation
system_log source

bdraco · 2024-04-01T22:40:43Z

I thought it was system_log but its not

Going to have to patch traceback

bdraco · 2024-04-01T22:43:16Z

Sadly I think I'm doing the path of another leak and not the original one being reported here

bdraco · 2024-04-01T22:51:48Z

2024-04-01 12:51:12.803 CRITICAL (SyncWorker_10) [homeassistant.components.profiler] New object traceback (0/23) at /opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py: <traceback object at 0x2de056ac0>
2024-04-01 12:51:12.803 CRITICAL (SyncWorker_10) [homeassistant.components.profiler] New object traceback (0/23) at /opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py: <traceback object at 0x2de057900>
2024-04-01 12:51:12.803 CRITICAL (SyncWorker_10) [homeassistant.components.profiler] New object traceback (0/23) at /opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py: <traceback object at 0x2da016f80>
2024-04-01 12:51:12.803 CRITICAL (SyncWorker_10) [homeassistant.components.profiler] New object traceback (0/23) at /opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py: <traceback object at 0x2eb82d840>
2024-04-01 12:51:12.803 CRITICAL (SyncWorker_10) [homeassistant.components.profiler] New object traceback (0/23) at /opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py: <traceback object at 0x2eb3958c0>
2024-04-01 12:51:12.818 CRITICAL (SyncWorker_10) [homeassistant.components.profiler] New objects overflowed by {'traceback': 18}

* Add missing platforms_exist guard to check_config related issue #112811 When the exception hits, the config will end up being saved in the traceback so the memory is never released. This matches the check_config code to homeassistant.config to avoid having the exception thrown. * patch * merge branch

home-assistant bot added the integration: homeassistant label Mar 9, 2024

bdraco added the memory-leak label Mar 9, 2024

bdraco mentioned this issue Apr 1, 2024

Add missing platforms_exist guard to check_config #114600

Merged

20 tasks

bdraco added the integration: system_log label Apr 1, 2024

bdraco removed the integration: system_log label Apr 1, 2024

bdraco mentioned this issue Apr 1, 2024

Fix memory leak when importing a platform fails #114602

Merged

20 tasks

balloob closed this as completed in #114602 Apr 2, 2024

github-actions bot locked and limited conversation to collaborators May 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Yaml reload causes memory % increase which is not released #112811

Yaml reload causes memory % increase which is not released #112811

Mariusthvdb commented Mar 9, 2024

home-assistant bot commented Mar 9, 2024

schellevis commented Mar 17, 2024 •

edited

Loading

erkr commented Mar 17, 2024 •

edited

Loading

Mariusthvdb commented Mar 17, 2024

erkr commented Mar 17, 2024 •

edited

Loading

schellevis commented Mar 18, 2024

Ftown14 commented Mar 19, 2024 •

edited

Loading

joostlek commented Mar 19, 2024

home-assistant bot commented Apr 1, 2024

bdraco commented Apr 1, 2024

bdraco commented Apr 1, 2024

bdraco commented Apr 1, 2024

Yaml reload causes memory % increase which is not released #112811

Yaml reload causes memory % increase which is not released #112811

Comments

Mariusthvdb commented Mar 9, 2024

The problem

What version of Home Assistant Core has the issue?

What was the last working version of Home Assistant Core?

What type of installation are you running?

Integration causing the issue

Link to integration documentation on our website

Diagnostics information

Example YAML snippet

Anything in the logs that might be useful for us?

Additional information

home-assistant bot commented Mar 9, 2024

schellevis commented Mar 17, 2024 • edited Loading

erkr commented Mar 17, 2024 • edited Loading

Mariusthvdb commented Mar 17, 2024

erkr commented Mar 17, 2024 • edited Loading

schellevis commented Mar 18, 2024

Ftown14 commented Mar 19, 2024 • edited Loading

joostlek commented Mar 19, 2024

home-assistant bot commented Apr 1, 2024

bdraco commented Apr 1, 2024

bdraco commented Apr 1, 2024

bdraco commented Apr 1, 2024

schellevis commented Mar 17, 2024 •

edited

Loading

erkr commented Mar 17, 2024 •

edited

Loading

erkr commented Mar 17, 2024 •

edited

Loading

Ftown14 commented Mar 19, 2024 •

edited

Loading