New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Utility Meter: Invalid state #90914
Comments
Hey there @dgomes, mind taking a look at this issue as it has been labeled with an integration ( Code owner commandsCode owners of
(message by CodeOwnersMention) utility_meter documentation |
I'm seeing the same thing now since 2023.4
|
Have same warnings in log since installing of 2023.4
|
this happens on each restart, but, if based on template sensors at least, also on each template reload, hence the 'unavailable'.
|
This is not an error message... this is a warning. It means that utility meter is unable to fully track consumption values from the source sensor because the source sensor is either Utility Meter continues to work... it just skipped a some state changes from these invalid states to good states... The warning is there, so that you know that the source sensor needs to be fixed... |
That's what I reported in the issue. This warning wasn't there before the upgrade to 2023.4 and I think it's quite usual, that many sensors are not available at startup or am I wrong? I can live with this log message but there may be many users that are worried about this. |
I have tons of warnings here: #90930 and hope this can be fixed. |
Hmm, that issue is not related to this one, your utility meter just doesn't have a value, you need to fix that... Or just disable it, since it doesn't work anyway:-) The issue here is only 1 warning after restart HA, when integrations are not yet ready , when integration is loaded, the warning stops |
Why you think so? My sensors still working, but with a warnings, same situation for other users. |
There must be one sensor with an unknown value, check that one.. also your log error is different, and we only see it one time , not spammed, so your utility meter is just not working, but best discuss this in your issue, it's not related to this one |
You are wrong, but kk, discussion stopped. |
I believe we can rule out "non working meters" given the amount of users reporting the issue starting exactly after moving to 2023.4.0. It might be that at initialization there is no value yet which results in the error. I however do not recall seeing this warning earlier, while now - after upgrading to 2023.4.0 and also 2023.4.1 I have similar records in my log:
The underlying logic in my setup is that I create sensors first (based on my integrations / meters) after which the utility_meters are created for my dashboards:
The same actually also causes an error on my side that I believe is new (one time at (re)start and everything works fine afterwards):
In my view, something "structurally" changed causing the warnings and errors. This might require a code fix or changes by people using the functionality: I leave it up to the experts to indicate what's is needed, but I hope this more elaborate scenario helps to narrow down on the RC :). |
Same here, these warnings after each restart since 2023.4.0:
In my opinion, a warning like this should contain the entity_id of the utility meter causing it! Otherwise how to correct any issues, if you don't know what entity has problem?
sensor.fridge_energy here is provided by a Zigbee plug through Z2M, so I've changed this device to send retained MQTT messages, and it seems to cleared the issue, so I guess this happens when MQTT is already started but Z2M not yet, so no value for sensors like this. |
The WARNING (not error!) message was indeed introduced in release 2023.4 The objective was for users to understand that their utility meter is skipping readings due to mis behaving source sensors. The best approach is to fix the SOURCE sensor, from various issues z2m looks to be a recurring culprit ... |
CC @Wesley-Vos |
Yes, but
|
Thanks for pushing the improvement in the logging @dgomes . I do however also agree with @realthk on the second comment: in my case the source (not being Z2M) is only "misbehaving" at the (re)start / (re)load of Home Assistant and unfortunately at every (re)start / (re)load. After all has been initialized there is no issue anymore (and if there would be an issue, then I would like to see it in the log). With the current setup, warnings could potentially be seen as "spam to skip" in the logs. |
updated the PR to only activate warnings after HA is marked as started... |
What about the template reload ? Will we be safe there too now? |
This is not a fix... this is a ignore until HA is marked as started |
I've added yet another PR that might address the template issue |
thx Diogo, hope it will see to it. appreciated! btw, since trigger based template sensors retain their state during restarts, would it be an idea to rewrite the used (regular) template sensors to trigger based template sensors? In the hope these states wont go unavailable during these essential moments. |
@Mariusthvdb that would be a different issue |
I'm seeing the warning about unavailable entities for several of my entities coming from an esphome device which I'm guessing is due to the integration not having the entities available on time. But I see the PR added to wait for HA to have started so I'm not sure why I still get those warnings. |
updated to 2023.4.2 now and the PR changed the Warning message to include the entities in play, which is really good, thanks!:
ofc, I would still hope these could be prevented too, as the appear on template reload.... |
Please wait for 2023.5.3, then report back |
Aah ok, nice. I was not aware that a (potential) fix is coming. Thanks for mentioning. |
I've installed 2023.5.3. The same error appears when restarting Home Assistant. |
For esphome devices in which you might trust their reliability you should use |
I've set periodically_resetting:true because the esphome device sensor will go back to 0 when i reboot the esphome device. That being said, i am unable to understand how this is impacting the issue at hand. The issue that i am facing, is that a reboot of home assistant generates the warning "an invalid state change coming from sensor.gang_watermeter_totaalverbruik." The esphome device itself is running nicely in the background. In my case, it is a water meter which is only giving a pulse once every half hour or so (whenever i use 1 liter of water). In practice, this means that the sensor value of the esphome device stays unchanged while home assistent is restarted. Yet, the warning appears in the logs of home assistant. |
The problem is that HA does not know if your water meter made a change while it was unavailable, therefore in the next reading coming from the meter the utility_meter will either skip that reading (the default solution, which leads to missing values) or it will take into consideration the previously recorded state (before the meter went unavailable - that's the Previous to the introduction of this option, this issue was not logged, basically because there was no alternative... (silent error) sure we can remove the warning message, but you will miss some important (?) readings and wonder what's wrong (nothing is wrong, the utility_meter just doesn't have enough information to deal with a sensor that came back from unavailable) |
The sensor itself is fully working and rock-solid. The issue the booting of Home Assistant itself. By definition, when home assistant is starting, sensors will often be unavailable until home assistant is fully started. This leads me to a different conclusion: the utility meter is not correctly handling the startup of Home Assistant. |
Some more clarification, It is to be used by devices that reset during unavailability (those unreliable sensors that loose power/connection often) The default behaviour for several years was to consider sensors as unreliable and just ignore their first state transition from unavailable to available, effectively only tracking changes while HA is working. |
I'm aware of that :) My sensor is designed to jump back to 0 when i shut it down from power. That happens sometimes (like once every month or so, whenever i have an electrical job to do at home). That is expected behaviour. And utility meter is handling that scenario perfectly. The issue is imho the warning message itself, which is not recognizing that home assistant is still in startup phase. |
The best scenario would be that:
|
I think some of the issues that I discussed in #72943 (my last comment for example) can be related. The In addition I noted that in 2023.5.3 I cannot read anymore any utility meter if the source sensors are unavailable, because they also turn unavailable. This is rather problematic and never occurred before. A common use case where this is evident is having a utility meter attached to a solar inverter: when the inverter shuts off at night you cannot read the meter measuring daily, weekly or yearly energy. |
@gfrancesco good point, I've reverted the unavailability of utility_meter in #93274 |
Would that also fix the warning "an invalid state change coming from sensor.xxxxx" when rebooting home assistant? |
Sorry guys, but #93274 got closed You will need to find a fix in which your inverter component doesn't go unavailable 🤷♂️ |
I think there are 2 puzzles at hand: |
I appreciated the effort @dgomes, might be that the goal of the PR was misunderstood?
In my opinion, it doesn't make much sense, especially considering that this is not an edge case at all. Source sensors go unavailable all the time, that's the norm not an exception, and it was correctly handled for years! This is further aggravated if your source sensor is battery driven and just measures intermittently, imagine water consumption for a sprinkler for example. To save battery the source sensor would be unavailable most of the time and the water consumption would be readable only when the sprinklers are on, which would not be very useful. Even if you think how physical utility meters works, it doesn't make any sense. Imagine if you cannot read how much water you used this year just because the little plastic spinner of the sensor broke. |
Me too, thank you dgomes for all the help and effort you put into this. |
There has been a long issue with what happens to helpers when their sources are unavailable, and it made sense to me to make the utility_meter unavailable when it's source is unavailable. From this issue discussion I realised that my previous judgement was not correct and therefore created a new PR to revert not the handling of unavailable sources but the unavailability of the utility_meter itself - this last PR was closed. |
Do you have a link to the issue you are referring? |
It's not a single issue, you can search this repository issues. |
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. |
Hi, the problem still occurs :( I'm searching for a workaround for this issue, because currently I can't see all stats on the dashboard after sunset. I've tried to create a separate template sensor that is set to 0 instead of unavailable but this causes wrong calculations on the utility meter. The values get doubled/tripled during the day (probably when the inverter loses connection). Can we get any info if this problem will be resolved or can we use some other workaround to fix this issue? |
It has been made clear that, according to the dev team, this is not a bug. It is a behavior that has been introduced deliberately. See my posts above, anyway I won't hold my breath for this to be changed since the PR was closed. At the moment I'm using a partial workaround using trigger template sensors. Essentially the source sensor used by the utility meter becomes the trigger sensor. The trigger sensor references the energy meter of your inverter, and it only updates when there is a new valid read coming from your inverter. For example if your inverter goes down and its energy meter reads unavailable, the trigger sensor is not updated, does not go unavailable, and keeps reporting the latest number, thus the utility meter also keep showing a good read. Here's an example of one of my trigger sensors: - trigger:
- platform: state
entity_id: sensor.inv_1_e
not_to:
- "unavailable"
- "unknown"
- "none"
sensor:
- name: "Inv-1 E NU"
state: "{{ states('sensor.inv_1_e') }}"
state_class: "total_increasing" |
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. |
The problem
Since I upgraded to 2023.4 I get theses warnings in the logs:
What version of Home Assistant Core has the issue?
core-2023.4.0
What was the last working version of Home Assistant Core?
2023.3.6
What type of installation are you running?
Home Assistant Container
Integration causing the issue
Utility Meter
Link to integration documentation on our website
https://www.home-assistant.io/integrations/utility_meter
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Additional information
No response
The text was updated successfully, but these errors were encountered: