New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Template editor hangs Home Assistant #40621
Comments
The system is likely hanging because the template is listening for all state changed events when we can't successfully evaluate a template as optimistically waiting for the template evaluation to be successful. Normally this isn't an issue but it seems like there is something on your instance that is generating more state changed events than the template engine can keep up with which leads to it being overwhelmed. |
I added functionality in #40624 to trap all the jinja2 errors which will prevent listening for all states on error. Regardless, the system should have been able to keep up with that so I suspect you have something else going on as well that might be brought to light with a |
This comment has been minimized.
This comment has been minimized.
chief, this templates were working just fine on all my instances up to 114.4. And only in 115 it cause the system to be non-responsive at all (cross tracking from other instances showing that it's offline). Only diff from "general" system that i'm having up to 2500 entities in my system. this how it looks like from portainer - and it's not dropping down when i do delete the template and press the Reset to Demo ... it might be hat my templates not the best ones but they were working just fine even on RPI3b+ and this is happening on Xeon system as well as on i3 or celeron...my RPI simple not able to restart container - only power cycle helps... |
In 0.114.4 we were not capable of tracking all state changed events from a template so you would not get any listeners when using
How many state changed events / second does the system fire? |
But it were giving correct answer all the time without any problems. and adding antity_id we were able to set "update interval"
according to my postgre stats - like 15-20 event per seconds. additionally once system starts evaluation template shown at the top, connection to addons got lost in couple of minutes - i was installing py-spy in terminal addon and get connection closed message. no any reaction on menu\buttons\triggers whatever. only container restart. |
I'm able to replicate the issue with:
|
I can't replicate the issue with the template provided in the example using a system with 1200 states on the busiest system I have. The number of state changes events per second is much much lower than the reference system:
|
so what can be done? |
At 16.18 state changes per second that is ~1.4 million state changed events fired per day. That's an impressive amount of state changes. That means the template needs to check 3.5 billion states per day (in this case its actually 7 billion since its iterated twice). Listening and iterating all states is never going to be viable on a system with those numbers no matter how much we optimize templates. The only solution I can come up with is to revert the change that added listeners for all states but that isn't going over very well as other users rely on this. |
maybe optionally add entity_id back - if it's specified use it (114 style), if not - than new 115 style |
The template tracking is used in bayesian sensors, template entities, automation triggers, universal media players, and the websocket api (the case you are reporting). This would handle the template entities case but it doesn't solve the issue you are reporting. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
it will. my system will not hang on template testing. as of now - i need to restart container each time when i have an error. plus - to be honest - do we really need this "feature"? it was working just fine in the past. and not it's enchanted meaning broken. |
jinja doesn't have a timeout capability. Since we have to render templates in async we don't have an easy way of aborting the execution without a signal which may not be safe to do.
If we implemented that we would end up with multiple template implementations in template entities. This issue isn't about template entities though its about the template editor hanging in developer tools. |
but it will became about template templates in general once a user will create this kind of template. dev tool is just on the top of all that |
I'm come back to this once #40272 is reviewed as it will give more flexibility on how we can handle this without breaking state counting. |
Assuming it solves all the cases, disabling the all states listener by default and implementing a flag |
This comment has been minimized.
This comment has been minimized.
Restoring entity_id would require the user to know about the flag in advance instead of trying to figure it out after they already have a performance issue. It also doesn't solve this issue. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Since allow_all_states_listener doesn't seem to be desired, I'm going to hold off moving forward until we can come up with a solution that accommodates the use cases without implementing two systems and solves the issue submitted here. |
This comment has been minimized.
This comment has been minimized.
Please do not spam this issue with discussion about entity_id being brought back. I've hid these comments. Next occurrence will result in a ban. This is the issue tracker, not for feature requests. The only thing you guys are achieving is that you are harassing someone who is actually trying to solve the problems. That's not something we accept in this community. |
Within the scope of the dev tools we need some type of safety net since users need to be able to safely experiment that covers:
Example: massive iteration
Example: iterate all states on busy systems (with the reference point being 1 million state change events per day or 3.5 billion iterates)
Example: any invalid template.
|
Why was this hidden? It was on the exact issue of this issue tracker my contribution was aimed. A template killing the instance in dev template tools Please unhide? |
Penny had some thoughts on this while working on the new template engine: home-assistant/architecture#206 – maybe it's better to continue the point 2 discussion there. |
Fantastic 👍 🥇 💯 🚀 This is exactly the help I was looking for 😄 😄 😄 |
@bdraco This may be related, I've noticed that listeners are set to all states when using |
@Petro31 |
Great, thanks |
The problem
testing templates in editor in 115.0-115.3 very tricky -making mistake in template you are getting hanging system, only container restart via portainer helps. below you can find log at the moment of testing. I think test environment should not lead to system reboot at any case
Environment
Problem-relevant
configuration.yaml
evaluating this kind of templates leads to system hangs (even not able to save config files from IDE addon)
Traceback/Error logs
Additional information
The text was updated successfully, but these errors were encountered: