Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sensors 'unavailable' after reboot #33

Open
rowebil opened this issue Mar 1, 2021 · 48 comments
Open

Sensors 'unavailable' after reboot #33

rowebil opened this issue Mar 1, 2021 · 48 comments

Comments

@rowebil
Copy link

rowebil commented Mar 1, 2021

All of my Sensors show 'unavailable' after I reboot the Home Assistant. It seemed like this was fixed in a prior bug patch release but appears not.

I usually have to remove the integration from everywhere and then re-add it. This takes so long - sometimes hours until it finally asks me to sign into Home Assistant. :(
@mpatl
Copy link

mpatl commented Mar 1, 2021

Same. Working yesterday. Not working in the morning.

Love the integration, too bad Emporia doesn't embrace it.

@magico13
Copy link
Owner

magico13 commented Mar 1, 2021

Some other people have seen them disappear somewhat randomly as well. you can try another reboot since that basically sets up the integration from scratch using the stored settings, ie it'll log in and try to set up all the sensors again. If there's an error during the set up it doesn't have any retry logic built in at the moment. That might be something I can look into, or periodically checking if there are new sensors so you wouldn't need to reboot if you added a new plug or it encountered an error earlier, or something.

@skynet01
Copy link

skynet01 commented Mar 5, 2021

anyone else experiencing this? I rebooted a few times now and none are coming up

@rowebil
Copy link
Author

rowebil commented Mar 5, 2021 via email

@mpatl
Copy link

mpatl commented Mar 5, 2021 via email

@skynet01
Copy link

skynet01 commented Apr 3, 2021

Is there a more permanent solution then rebooting? It seems sometimes it doesnt come up after the reboot so i have to reboot twice, and sometimes it just stops working randomly. Any way to create a more permanent retry if it fails to connect?

@rowebil
Copy link
Author

rowebil commented Apr 3, 2021 via email

@rowebil
Copy link
Author

rowebil commented Apr 3, 2021 via email

@skynet01
Copy link

skynet01 commented Apr 3, 2021

can you describe this solution with the error correction that you implemented? Would like to set that up as well

@rowebil
Copy link
Author

rowebil commented Apr 3, 2021 via email

@magico13
Copy link
Owner

magico13 commented Apr 4, 2021

@rowebil Would you mind sharing what you made/changed? I definitely wouldn't say no to a pull request but I also understand if you'd rather not share it.

@magico13
Copy link
Owner

magico13 commented Apr 4, 2021 •
edited

Something I just realized exists that you can use instead of restarting is just going to the Integrations page, selecting the 3 dots on the Emporia Vue integration and then selecting Reload. That will run through the setup code again so it should reauth and populate the devices list again.

image

Edit: This seems to only exist in my development environment. Might be something that is coming soon. Either way, I'm working on trying to move a bunch of the setup so I can take advantage of the automatic retry that home assistant has built in if a component fails to go through setup. I pushed an update to master with that change that was working fine in my development environment but doesn't seem to work at all in my personal setup, still debugging it.

@skynet01
Copy link

skynet01 commented Apr 6, 2021

Thanks for the update on this! I do have the reload option available and i don't have dev envirment but it only works sometimes just like rebooting.

@magico13
Copy link
Owner

magico13 commented Apr 11, 2021 •
edited

@skynet01 Would you be willing to try to install the "master" version? Just go into HACS and press reinstall and then from the drop down choose master. I reworked a bunch of the initialization so it should use the built-in retry logic if it has an error during setup and there's some additional logging that might be useful. With a successful setup you should see some messages like this in the logs page.

image

@skynet01
Copy link

It's installed through a master version as well. It works fine for some time and then it just goes out like everyone is mentioning here.

@magico13
Copy link
Owner

Are there any error messages in the logs that you can share? I don't have the same issue, usually for me if a particular request fails it comes back up just fine at the next call, so the sensors will say unavailable for up to a minute or two before coming back on their own. Individual update failures shouldn't result in any permanent issues, it's only during the initial setup that the calls have to succeed so that all of the sensors are added correctly so that's what I've been focusing on fixing.

@skynet01
Copy link

I get these errors:

first this one:
2021-04-13 10:48:05 WARNING (MainThread) [homeassistant.loader] No 'version' key in the manifest file for custom integration 'emporia_vue'. As of Home Assistant 2021.6, this integration will no longer be loaded. Please report this to the maintainer of 'emporia_vue'

followed by these two a few minutes later
2021-04-13 10:53:00 WARNING (MainThread) [custom_components.emporia_vue.sensor] No channels found during update 2021-04-13 10:53:00 ERROR (MainThread) [custom_components.emporia_vue.sensor] No data found for 1 second updater

@magico13
Copy link
Owner

First one isn't an issue and will be fixed in the next update (master branch already has that fixed). The latter ones are the fun ones. I'm not sure why the update wouldn't return proper data but it could be due to local network connectivity issues, authentication token expiration, issues server-side, or maybe rate-limiting. Since that's the 1 second updater it should run again 60 seconds later and would hopefully get data then. How many channels/devices do you have?

@skynet01
Copy link

I just have 8 channels, I know I wish it was a less general error message.

@skynet01
Copy link

I think there must be something else going on, as it shows it finds my emporia devices it just doesn't update them. Running on your latest master version.

10:08:01 AM – (WARNING) Emporia Vue - 1s Update data: {}
10:08:01 AM – (WARNING) Emporia Vue - No channels found during update
10:08:00 AM – (WARNING) Emporia Vue - Found 2 Emporia devices

@magico13
Copy link
Owner

Someone mentioned a similar issue on the PyEmVue github. I'm not sure yet if it's my code or their API not returning the data on the first call. Definitely looks similar magico13/PyEmVue#14

@skynet01
Copy link

What's interesting is that I have 9 total devices (the main one and 8 channels) and it only says found 2 devices. But it hasn't worked for about a week now, with no amounts of restarts fixing it. Let me know if there is anything i can do to help you trouble shoot this further

@magico13
Copy link
Owner

Regarding devices vs channels: the log message shows the number of "devices" as defined by the API, which is physical devices, in your case the main one and the expansion, whereas within home assistant I assign a "device" per channel. I've updated the message to indicate the number of total channels found as well, eg Found 3 Emporia devices with 10 total channels

I also pushed an update where if an update for the channels fails it'll try again right away. I also made it check for one second in the past instead of the current time in case it's a small timing mismatch. I'm really not sure why it'd be returning the devices just fine but the usage is coming back empty. I might ask you to do some tests with the PyEmVue library directly.

Also what timezone are you in? I know someone mentioned issues with BST (#35) so I wonder if that's somehow related. I'm in EDT for comparison.

@skynet01
Copy link

I am in MST, is the new updated on the master branch? I can try testing it as well.

@skynet01
Copy link

skynet01 commented Apr 23, 2021 •
edited

Well with the latest update the sensor is working again ❤️.
Hope it holds this time :)

I do get the following 2 warnings, but it looks like like its just debug information:
...and maybe an incorrect number of devices? I assume 17 channels are the entities that are created (seconds and minutes)

2021-04-23 00:28:10 WARNING (MainThread) [custom_components.emporia_vue] - Found 2 Emporia devices with 17 total channels

2021-04-23 00:28:11 WARNING (MainThread) [custom_components.emporia_vue] -  1s Update data: {'15928-1,2,3-1S': {'device_gid': 15928, 'channel_num': '1,2,3', 'usage': 11855, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa8aa7ce8>}, '15928-1-1S': {'device_gid': 15928, 'channel_num': '1', 'usage': 2165, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}, '15928-2-1S': {'device_gid': 15928, 'channel_num': '2', 'usage': 26, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}, '15928-3-1S': {'device_gid': 15928, 'channel_num': '3', 'usage': 0, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}, '15928-4-1S': {'device_gid': 15928, 'channel_num': '4', 'usage': 0, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}, '15928-5-1S': {'device_gid': 15928, 'channel_num': '5', 'usage': 141, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}, '15928-6-1S': {'device_gid': 15928, 'channel_num': '6', 'usage': 262, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}, '15928-7-1S': {'device_gid': 15928, 'channel_num': '7', 'usage': 60, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}, '15928-8-1S': {'device_gid': 15928, 'channel_num': '8', 'usage': 7, 'scale': '1S', 'info': <pyemvue.device.VueDevice object at 0xa4a5e2c8>}}

@skynet01
Copy link

skynet01 commented Apr 23, 2021 •
edited

...And it died last night, and restarting didn't help either

this is what it's doing during restart

10:25:17 AM – (WARNING) Emporia Vue - 1s Update data: {} 10:25:17 AM – (WARNING) Emporia Vue - No channels found during update for scale 1S
10:25:17 AM – (WARNING) Emporia Vue - No channels found during update for scale 1S. Retrying...
10:25:16 AM – (WARNING) Emporia Vue - Found 2 Emporia devices with 17 total channels

If you would like i can give you my login credentials so you can check it on your end?

@magico13
Copy link
Owner

If you don't mind that I can certainly try using your credentials to see if it happens on my machine as well. I'm not sure if there's a way to directly message people on GitHub so you can email me directly at magico1313(at)gmail.com. I don't think a restart is going to fix it since the initial calls all went through, it's just the individual updates failing.

Hmm, I wonder if the signal between your Vue and your router is weak so it's not updating once a second. If you swap off of the 1 second sensor and use the 1 minute sensor does it still have issues? Does anything look weird in the app when you look at the 1 second history (missing data, or data that's repeated [ie a flat line])?

@rowebil
Copy link
Author

rowebil commented Apr 23, 2021 via email

@skynet01
Copy link

Email with creds sent, I have a good connection and I have a minute sensor as well that also doesn't work. There are no gaps in history in the app (you can also login there and see what it sees). I do have solar panels so it shows negative numbers sometimes but I dont think this would do it.

@magico13
Copy link
Owner

magico13 commented Apr 24, 2021 •
edited

Got it. I won't get a chance until this evening to look at it but I should have some time then to really dig into it.

Edit: Left it running for a few hours in my dev environment with no empty updates at all/nothing going "unavailable". I'm going to try to do some more tests tomorrow to see if I can make it misbehave. It's definitely cool to see how it works with solar panels though.

@skynet01
Copy link

skynet01 commented Apr 25, 2021 •
edited

mine started working again last night randomly, which never happened before, I usually had to restart it. I am starting to think maybe the issue is on their side? The only thing I noticed is this in the logs and it shows up a few times. But it still works
Error fetching sensor1s data: Error communicating with Emporia API: 500 Server Error: Internal Server Error for url: https://api.emporiaenergy.com/AppAPI?apiMethod=getDevicesUsage&deviceGids=15928&instant=2021-04-25T16:08:04.003812Z&scale=1S&energyUnit=KilowattHours

@magico13
Copy link
Owner

The 500 error definitely looks to be something on their side since I saw that both in my dev environment logged in as you and my personal one at the same time. I'm going to try to trigger a timeout or delay to see if I can get it to return no data like you were seeing. I've tried adjusting the clock to see if a clock drift would trigger it (asking for data in the slight future, getting no results) but that didn't cause any issues.

@skynet01
Copy link

yeah, it's been running ok pretty much from the time I gave you the credentials :) I am beginning to think that maybe they have some sort of API limit that got reset? Thank you again for testing this, you should set up donation button somewhere :)

@skynet01
Copy link

skynet01 commented Apr 30, 2021 •
edited

It's been working fine for about a week now until last night. I am beginning to think that the issue might be on their end? Rebooting or reloading also doesn't fix it. The error is different though:

Logger: homeassistant
Source: custom_components/emporia_vue/sensor.py:83
First occurred: April 29, 2021, 2:22:24 PM (339 occurrences)
Last logged: 5:03:54 AM

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 130, in _handle_refresh_interval
    await self.async_refresh()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 161, in async_refresh
    await self._async_refresh(log_failures=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 235, in _async_refresh
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 295, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 310, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 334, in _async_write_ha_state
    sstate = self.state
  File "/config/custom_components/emporia_vue/sensor.py", line 83, in state
    usage = self.coordinator.data[self._id]["usage"]
KeyError: '15928-1,2,3-1S'

This was referenced May 6, 2021
Closed
Closed
@skynet01
Copy link

it stabilized for now, usually reloading the component fixes the issue

@haddood
Copy link

haddood commented Jul 4, 2021 •
edited

similar issues after the last version 0.5.0 ... the 1_min sensors are not loaded while the 1_D and 1_mon are loaded normally after a restart.
currently I added an automation to reload the integration after 10 seconds from restart. no errors in the log
HASS 2021.6.6

@rowebil
Copy link
Author

rowebil commented Jul 4, 2021 via email

@magico13
Copy link
Owner

magico13 commented Aug 7, 2021

This seems to have stabilized, correct? Are we good to close this issue and just open a new one if issues keep coming up, especially if a reload doesn't fix it?

@skynet01
Copy link

skynet01 commented Aug 7, 2021

It's stabilized for now although once or twice a week other channels stop receiving and reload is needed.

@bwbuford
Copy link

I just want to comment that this issue is still happening for me in 0.6.2. Only my "min" entities are (seeming randomly) becoming unavailable; the daily and monthly continue to work fine.

This happens nearly daily, and sometimes it takes multiple 'reloads' of the integration to get them to come back.

@skynet01
Copy link

How hard would it be to implement a logic that checks if some sensor has become unavailable for say 5 minutes and then just reload the component? I wish there was a service call to do that then i could manually set this up using automation.

@magico13
Copy link
Owner

How hard would it be to implement a logic that checks if some sensor has become unavailable for say 5 minutes and then just reload the component?

There's no easy way that I've seen so far. I have no visibility into the Home Assistant setup and there's no method I know of to programmatically reload the integration, especially from within the integration.

They way this works is that Home Assistant calls some methods to initialize the integration and that's when I do the log in and get all the different channels, then I return a list of sensors to Home Assistant. Then Home Assistant calls the update methods I defined during startup and that updates the sensor states. I don't really control any of that flow, I just make an Http request each time HA tells me to update and I've already got code there to retry if it gets no data and to print an error if the retry fails.
I don't know of any way to tell Home Assistant to reload from the beginning and I apparently can't tell very well if the sensors have gone unavailable. I'm not sure what I'd even do then because if it's the http request failing then maybe I'd try forcing a new log in to the API but if the sensor itself is broken I might not be getting called by HA to update in the first place.

@skynet01
Copy link

skynet01 commented Aug 18, 2021 •
edited

I figured out how to automate the reloading of an integration. You can do this for any other integration as well.

Set up automation like this:

- id: auto_reload_emporia
  alias: "Keep Emporia running" 
  mode: single
  description: 'Check if Emporia is down'
  trigger:
  - platform: state
    entity_id: sensor.power_av_closet_6 # use any entity that is an emporia device / channel 
    to: unavailable
    for: 00:05:00
  - platform: template #check if any sensor gets stuck on the same number for 10 min
    value_template: "{{ as_timestamp( utcnow() ) | int > as_timestamp( states.sensor.power_pelican_123.last_changed ) | int + 10*60 }}"
  condition: []
  action:
  - service: rest_command.reload_emporia
    data: {}

Then in your main config file

rest_command:
  reload_emporia:
    url: http://[IP]:8123/api/config/config_entries/entry/[entry_id]/reload 
    method: POST
    headers:
      authorization: 'Bearer [Long-Lived Access Token]'
      content-type: 'application/json'
  1. [IP] = home assistant ip
  2. [entry_id] = go to config/.storage/core.config_entries search for emporia_vue you will see entry_id as the first record
  3. [Long Lived Access Token:] = Token can be created in the HA menu under your instance name. Yes there is a space between Bearer and the actual token

@tteck
Copy link
Contributor

tteck commented Aug 26, 2021 •
edited

I was seeing this issue as well. It seems that the upcoming 2021.9.0 release has maybe solved it. After several reboots/restarts, this issue hasn't showed itself.

EDIT: Well, dang-it happened with 2021.9.0b1

@skynet01
Copy link

skynet01 commented Sep 3, 2021

Yeah it still happens for me once a day or so. For some reason, the 5 min trigger is not working in my code above. Not sure why, if anyone has any insight that would be great.

@skynet01 skynet01 mentioned this issue Sep 3, 2021
Open
@magico13 magico13 mentioned this issue Sep 5, 2021
Open
@zztopgtb
Copy link

For me all minute entities are unavailable after reboot. Once I reload the integration it works as expected.

@JohnMcLear
Copy link

I'm still experiencing this issue so I don't think it's been resolved either in HA or in the Integration.

@skynet01
Copy link

I use node-red to reload the integration it works pretty well now. Here is the code...

[{"id":"8a93ebd45796b533","type":"server-state-changed","z":"1169dafb.347525","name":"Emporio channel is down","server":"8a745406.799e98","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.power_av_closet_6","entityidfiltertype":"exact","outputinitially":true,"state_type":"num","haltifstate":"0","halt_if_type":"num","halt_if_compare":"gte","outputs":2,"output_only_on_state_change":false,"for":"5","forType":"num","forUnits":"seconds","ignorePrevStateNull":true,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"true","valueType":"bool"},{"property":"data","propertyType":"msg","value":"","valueType":"date"}],"x":190,"y":1340,"wires":[[],["3a18fca6f551e11e"]]},{"id":"00fcf99a23deb608","type":"api-call-service","z":"1169dafb.347525","name":"Reload Emporio Config","server":"8a745406.799e98","version":5,"debugenabled":false,"domain":"homeassistant","service":"reload_config_entry","areaId":[],"deviceId":[],"entityId":["sensor.power_pelican_123"],"data":"","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"first","x":790,"y":1380,"wires":[["c7b66907f042d2b7"]]},{"id":"2237f15227046410","type":"inject","z":"1169dafb.347525","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":480,"y":1420,"wires":[["00fcf99a23deb608"]]},{"id":"715ab5db5fde4670","type":"api-current-state","z":"1169dafb.347525","name":"Check if backup","server":"8a745406.799e98","version":3,"outputs":2,"halt_if":"0","halt_if_type":"num","halt_if_compare":"gte","entity_id":"sensor.power_av_closet_6","state_type":"num","blockInputOverrides":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"for":0,"forType":"num","forUnits":"minutes","x":1200,"y":1320,"wires":[[],["00fcf99a23deb608"]]},{"id":"c7b66907f042d2b7","type":"delay","z":"1169dafb.347525","name":"","pauseType":"delay","timeout":"20","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1000,"y":1380,"wires":[["715ab5db5fde4670"]]},{"id":"9b1037d5faa2f4f8","type":"delay","z":"1169dafb.347525","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"10","rateUnits":"minute","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"allowrate":false,"outputs":1,"x":600,"y":1360,"wires":[["00fcf99a23deb608"]]},{"id":"962b64eac6d61292","type":"server-state-changed","z":"1169dafb.347525","name":"Emporio channel is down","server":"8a745406.799e98","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.power_fridge_5","entityidfiltertype":"exact","outputinitially":true,"state_type":"num","haltifstate":"0","halt_if_type":"num","halt_if_compare":"gte","outputs":2,"output_only_on_state_change":false,"for":"","forType":"num","forUnits":"seconds","ignorePrevStateNull":true,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"true","valueType":"bool"},{"property":"data","propertyType":"msg","value":"","valueType":"date"}],"x":190,"y":1400,"wires":[[],["3a18fca6f551e11e"]]},{"id":"3a18fca6f551e11e","type":"api-current-state","z":"1169dafb.347525","name":"HA restarted?","server":"8a745406.799e98","version":3,"outputs":2,"halt_if":"1","halt_if_type":"num","halt_if_compare":"lte","entity_id":"sensor.home_assistant_uptime","state_type":"num","blockInputOverrides":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"for":0,"forType":"num","forUnits":"minutes","x":420,"y":1360,"wires":[[],["9b1037d5faa2f4f8"]]},{"id":"8a745406.799e98","type":"server","name":"Skynet Home Assistant - Local","version":2,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":30}]

Maybe some sort of built in timeout can be created? Like if the value is unknown for a sensor try to reconnect?

@magico13 magico13 mentioned this issue Jul 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@JohnMcLear @magico13 @skynet01 @haddood @tteck @rowebil @zztopgtb @bwbuford @mpatl