Nest Thermostat integration stops working #92927
Comments
|
Hey there @allenporter, mind taking a look at this issue as it has been labeled with an integration ( Code owner commandsCode owners of
(message by CodeOwnersMention) nest documentation |
|
Mine has recently also started to do the same thing. |
|
@MattHadfield113 can you also attach diagnostics for the thermostat when this is happening? |
|
My assumption is that the issue here is the server is returning invalid data over pubsub that makes it look like there are no longer any supported hvac modes. I think it would be helpful if you turn up the debug logging described here https://www.home-assistant.io/integrations/nest/#troubleshooting and then we can look at the updates returned from the server just from around the time the thermostat appears to turn off. |
|
I am also seeing problems with thermostat not responding to the ThermostatTemperatureSetpoint.SetRange command when submitted via the homeassistant Nest integration, however when manipulated via the native Nest mobile app, temperature can be set just fine. Earlier I was seeing a 429 resource exhausted response for every set temperature API call from the UI. I tried deleting the integration and creating a new oauth ID and new device access project. Now, I don't see those errors, but no change takes effect. I am running 2023.6.2 by the way and here is diagnostic from Nest integration config_entry-nest-776cc3d0f37c2b3f3c9262fdf8fc73ef.json.txt Example logs: When I make changes via the native Nest app, the temperature updates do come through and show on the UI, so that part is working: |
|
@clarsen appreciate the report though it seems like different symptoms than this issue. But it also looks like an issue with the nest API and not a home assistant bug. |
|
As I replied before, please file a separate issue as it's not related to the parent right? |
|
This is still happening intermittently and it would make sense that maybe the Nest API is having sporadic issues. However, reloading the integration seems to immediately fix it and it remains stable for several hours before it reoccurs. Is there a state that is established when the integration is first loaded that maybe gets lost if the API hiccups? Is there a way to automatically reload or reinitialize the integration if the Nest API sends a bogus or empty response? |
|
@ianwood Please see my reply #92927 (comment) asking you for additional troubleshooting information which is needed to decide a path forward -- we need to see examples of the update messages in order to know what is happening. |
|
Also seeing a different symptom, but similar possible root cause where invalid messages are published by the API in #96434 -- getting debug logs of the messages being published would be very helpful. |
|
I'm seeing the same issue as here and #96434. How can I provide the debug logs? I enabled debug logging on the integration. 
|
|
@domoritz Logs are in |
|
Here are my anonymized and filtered logs. |
|
[edited a few times for clarity] Integration appears to fail at 5am where the thermostat is indicated as being off which it was not. The failures continue indefinitely at that point until the integration is manually reloaded at 1:23pm and then everything works again. Last successful thermostat setting at 12:12am: 
The first indication of failure is 5am where thermostat is incorrectly shown as off. Integration restored manually by reloading it at 1:23pm. 
|
|
The problem line is: There You can see a few lines before it looks normal and the available modes includes The right approach here is to get Nest API to fix this error. A good next step is to file an issue at: |
|
I've opened an issue with Google here: Google Issue Tracker 291659614 I've noticed the issue appears to be related to a persistent state between HA and the Nest API. Once the error occurs, it will continue to occur until the integration is reloaded. Once the integration is reloaded, the error stops occurring. |
|
Thank you. I agree that it seems like also a bug in HA that the problem doesn’t get resolved once the API responds correctly. |
|
Oh no just to clarify, see my update above #92927 (comment) this is a google nest publisher issue, not a home assistant bug. The way the integration works is that it polls the API on startup, then uses the subscriber API to efficiently be notified about updates to devices immediately as soon as they happen -- rather than polling constantly and inefficiently. The nest API is incorrectly publishing a command that says: "Your thermostat does not support heating or cooling" and home assistant honors that. On restart, the API returns the right response. This is a nest publisher bug that is very important to be resolved. @ianwood your bug report includes the eco trait, which is fine. You need to include an example with |
|
Thank you for the clarification. Makes sense. Anyone else experience this issue, please feel free to share debug information as I did on Google Issue Tracker 291659614. |
|
Thanks @ianwood - I'm having same issue. For all that are seeing this, please go to https://issuetracker.google.com/issues/291659614?pli=1 and click the +1 in top right corner to upvote. This is how Google prioritizes reviewing/fixing issues. |
|
What's better than +1 is to include detailed bug reports with debug logs showing the incorrect messages published. |
Just fyi: I suspect whatever broke is now fixed. I had setup an automation to alert me when it happens and no issues over last week. If it does I’ll upload logs. Been in debug logging mode this entire time waiting to capture… |
Still broken for me. |
|
I have a suspicion about getting to a broken state when my router froze the other day and I lost connectivity. I'm not totally sure and a haven't had time to investigate or experiment. Just saying it here in case that helps anyone figuring out the situation. I know there is the google api bug already documented and I also gave my +1 there as well. |
You are welcome to file a separate issue if you get more info. this bug is tracking something very specific. |
@allenporter - I managed to capture the issue (it happened briefly again this morning). I pasted problem code with eventID info into the above Google issues report. Hope that helps. Thanks again for all your guidance! |
|
I'm still having issues with controlling nest via home assistant. I haven't looked at the logs recently, though. |
I know you directed to another user - but if helpful: I'm still experiencing the problem. I also haven't heard any follow-up from Google after sending logs: https://issuetracker.google.com/issues/291659614?pli=1 (I know that isn't your area, and you were attempting another fix, but just as an FYI) |
|
@dredjohn21 Can you clarify what home assistant version you're using and what error you see? Yes, just to be clear, i'm assuming nothing has changed on the google side, but wanting to explore what options there are for mitigation. (e.g. ignoring broken messages, or in case of out of order messages still trying to have things end up in the correct stateetc). There are probably a handful of issues here, so having data about exactly what is happening is helpful. |
Hi - sure: The symptoms are same as reported above. I can enable detailed logging again and copy/paste logs after next occurrence. It just happened again this morning, but I didn't leave logging on since previously reported to Google. [I have an automation looking for the issue and restarting the integration usually gets it working... I have a notification to alert when it has to do this and it happens about once a day; usually in the morning] |
|
Yes, i new error logs would be helpful. I am surprised its the same given that code was rewritten. |
|
(HVAC modes can no longer be corrupted by invalid pubsub messages. we now grab it on startup from the API then it never chagnes) |
Thanks for helping to resolve. I'll get the logs and share. Maybe it's a different issue leading to same experience. My automation is simplistic: Looking for when my thermostat goes to OFF state. That was the easiest way to previously detect the problem and resolve. We'll see what is leading to that soon... |
@allenporter it just happened again: You’ll notice the thermostat available modes were messed up (only showing OFF) which triggered an automation to reboot the integration and in this example it solved the problem and the right values now came through and integration started working fine again. Hopefully Google can solve soon. |
|
@dredjohn21 right but my point is the integration already won't change the HVAC modes when that happens, after a recent fix. What version of HA? what is the automation detecting? |
|
I'm now using: Looking at the recent activity it appears to be happening when I have automations set to change the temperature (e.g. when we wake up in morning, turn on the heat) It doesn't always happen when the automation changes temperatures... but it sometimes does... so maybe at the time it's setting temp it checks on available HVAC modes and when it sees "heat" isn't available it turns OFF? (that's the Google bug they hopefully will fix) My automation to alert me of problems was coded based on the thermostat turning OFF, which the logs above show happens when the available HVAC options are not coming through properly. |
|
Unsure if this is related but my nest integration has started getting stuck at initializing after a HASS/router restart. I've also restarted the nest itself a couple times to no avail. This message is flooding the log something fierce (we're talking >700 entries a minute). |
|
@aredon please file a new issue and include all relevant information in the issue template. |
|
There hasn't been much activity in here but I wanted to pop in and say I'm still having the same issue and have been for quite some time. I posted data from this morning into the google issue : https://issuetracker.google.com/issues/291659614?pli=1 |
|
I got emailed by Google people to ask for more logs and I sent them details months ago. Nothing has been fixed yet as far as I can see. |
|
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. |
|
This is still a problem. I can turn the fan mode on but not off. |
|
This is stil broken - google asked for logs months ago and never did anything. Very frustrating. Please don’t close. |
|
API is still bombing out at least a few times a day. I've written an exception handler in Node Red to at the beginning of my HVAC routines to verify the thermostat is actually in the right mode before trying to set it. If it is reported as off when it isn't supposed to be, it reloads the integration which fixes the issue without human intervention. But there are still automations that get caught out from time to time. Minor inconvenience but would be better if Google just actually fixed it. 
|
Could you please share how you are reloading only the Nest integration via node red? |
|
I gave up on waiting and bought an Ecobee which has been rock solid with local control. Sold my nest on ebay. |
|
I meant to do a lot more digging, but I just haven't had the time. But I found the following, which might explain why messages are being received out of order. I still think that needs to be resolved. Any attempt to throw away out of order messages is only making the integration not really understand the state of the Nest devices. At least, that's my "2 cents." So, here is where I believe the subscription is being created: Here is the code for convenience: message_retention_duration = Duration()
message_retention_duration.FromSeconds(DEFAULT_MESSAGE_RETENTION_SECONDS)
subscription_request = {
"name": subscription_name,
"topic": topic_name,
"message_retention_duration": message_retention_duration,
}
_LOGGER.debug(f"Creating subscription: {subscription_request}")
subscriber.create_subscription(request=subscription_request)And here is what I found for creating a subscription with ordering: Again, for convenience: def create_subscription_with_ordering(
project_id: str, topic_id: str, subscription_id: str
) -> None:
"""Create a subscription with ordering enabled."""
# [START pubsub_enable_subscription_ordering]
from google.cloud import pubsub_v1
# TODO(developer): Choose an existing topic.
# project_id = "your-project-id"
# topic_id = "your-topic-id"
# subscription_id = "your-subscription-id"
publisher = pubsub_v1.PublisherClient()
subscriber = pubsub_v1.SubscriberClient()
topic_path = publisher.topic_path(project_id, topic_id)
subscription_path = subscriber.subscription_path(project_id, subscription_id)
with subscriber:
subscription = subscriber.create_subscription(
request={
"name": subscription_path,
"topic": topic_path,
"enable_message_ordering": True,
}
)
print(f"Created subscription with ordering: {subscription}")
# [END pubsub_enable_subscription_ordering]Notice how this snipped includes |
|
I am familiar with that. As I said before, lets get more specific though about an ordering problem you are seeing and we can discuss solutions for it. Please record the set of messages you receive, showing the ordering problem that it is causing. I didn't want to get into the weeds of the implementation details here before which is why I suggested focusing on the specific problem you are seeing. I said before we're not doing anything special about delivery order, but that isn't exactly true. To be clear, it is normal and expected for messages to be received out of order. We are following the API guide here https://developers.google.com/nest/device-access/api/events: and it links to documentation about message ordering. So we follow the advice there https://github.com/allenporter/python-google-nest-sdm/blob/9646bde0b20c927f1f2e2009ce86ff4c2c12f3c4/google_nest_sdm/device.py#L296 and use timestamp to aid in reconciliation of event order. Regarding the ordering feature itself in pubsub my impression is the ordering documentation states Again, if you want to talk about message ordering then lets talk about specifics of out of order messages you are receiving that are causing problems as requested above. |
|
https://issuetracker.google.com/issues/291659614 has an update that a fix is rolling out to prodution for the issue where thermostates appear off, and they request you provide update there if you see this issue. |
|
Google team says "The team has successfully implemented a fix, which is currently being rolled out to the production environment. However, due to the complexity of the issue, it's possible that this fix may resolve the problem for some thermostat models while others may still experience it. If you're still encountering issues with the Thermostat mode randomly reporting "Off" status, we kindly ask for your assistance in troubleshooting by providing us with most recent logs (including timestamps and time zone information) along with the model information of your thermostats." I am closing since and you can eport status on https://issuetracker.google.com/issues/291659614 if you are seeing an issue. Thank you to everyone who took the time to report information to Google. This issue never affected me personally so it really needed your patience and support to resolve. |
The problem
Nest thermostat integration (using the Google Cloud API) works normally for several hours or even days but then fails and the integration needs to be reloaded to work again. It seems that the more recent versions of core have made this problem worse as I am now having to restart the integration almost daily.
Integration will show as "Off" with no other modes when it has failed. Reloading the integration restores the normal HVAC modes. Looking at the history for the device when the integration has failed, it will show that it was turned off at some point earlier although there was nothing commanding it to do so.
What version of Home Assistant Core has the issue?
core-2023.4.5
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
Nest
Link to integration documentation on our website
https://www.home-assistant.io/integrations/nest/
Diagnostics information
{
"home_assistant": {
"installation_type": "Home Assistant OS",
"version": "2023.4.5",
"dev": false,
"hassio": true,
"virtualenv": false,
"python_version": "3.10.10",
"docker": true,
"arch": "x86_64",
"timezone": "America/Los_Angeles",
"os_name": "Linux",
"os_version": "6.1.24",
"supervisor": "2023.04.1",
"host_os": "Home Assistant OS 10.0",
"docker_version": "23.0.3",
"chassis": "embedded",
"run_as_root": true
},
"custom_components": {
"frigate": {
"version": "4.0.0",
"requirements": [
"pytz==2022.7"
]
},
"hacs": {
"version": "1.32.1",
"requirements": [
"aiogithubapi>=22.10.1"
]
}
},
"integration_manifest": {
"domain": "nest",
"name": "Google Nest",
"after_dependencies": [
"media_source"
],
"codeowners": [
"@allenporter"
],
"config_flow": true,
"dependencies": [
"ffmpeg",
"http",
"application_credentials"
],
"dhcp": [
{
"macaddress": "18B430*"
},
{
"macaddress": "641666*"
},
{
"macaddress": "D8EB46*"
}
],
"documentation": "https://www.home-assistant.io/integrations/nest",
"iot_class": "cloud_push",
"loggers": [
"google_nest_sdm",
"nest"
],
"quality_scale": "platinum",
"requirements": [
"python-nest==4.2.0",
"google-nest-sdm==2.2.4"
],
"is_built_in": true
},
"data": {
"data": {
"name": "REDACTED",
"type": "sdm.devices.types.THERMOSTAT",
"assignee": "REDACTED",
"traits": {
"sdm.devices.traits.Info": {
"customName": "REDACTED"
},
"sdm.devices.traits.Humidity": {
"ambientHumidityPercent": 64
},
"sdm.devices.traits.Connectivity": {
"status": "ONLINE"
},
"sdm.devices.traits.Fan": {
"timerMode": "OFF"
},
"sdm.devices.traits.ThermostatMode": {
"mode": "HEATCOOL",
"availableModes": [
"HEAT",
"COOL",
"HEATCOOL",
"OFF"
]
},
"sdm.devices.traits.ThermostatEco": {
"availableModes": [
"OFF",
"MANUAL_ECO"
],
"mode": "OFF",
"heatCelsius": 15.555527,
"coolCelsius": 29.444397
},
"sdm.devices.traits.ThermostatHvac": {
"status": "OFF"
},
"sdm.devices.traits.Settings": {
"temperatureScale": "FAHRENHEIT"
},
"sdm.devices.traits.ThermostatTemperatureSetpoint": {
"heatCelsius": 20.555555,
"coolCelsius": 23.333334
},
"sdm.devices.traits.Temperature": {
"ambientTemperatureCelsius": 20.839996
}
},
"parentRelations": [
{
"parent": "REDACTED",
"displayName": "REDACTED"
}
]
},
"command": {
"sdm.devices.commands.ThermostatMode.SetMode_count": 1,
"sdm.devices.commands.ThermostatMode.SetMode_sum": 755,
"sdm.devices.commands.Fan.SetTimer_count": 1,
"sdm.devices.commands.Fan.SetTimer_sum": 771
},
"event_media": {
"event": 6
}
}
}
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 199, in handle_call_service await hass.services.async_call( File "/usr/src/homeassistant/homeassistant/core.py", line 1820, in async_call task.result() File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in _execute_service await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)( File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 216, in handle_service await service.entity_service_call( File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 798, in entity_service_call future.result() # pop exception if have File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call await coro File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call await result File "/usr/src/homeassistant/homeassistant/components/nest/climate_sdm.py", line 287, in async_set_hvac_mode raise ValueError(f"Unsupported hvac_mode '{hvac_mode}'") ValueError: Unsupported hvac_mode 'heat_cool' 2023-05-09 19:30:00.195 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [139724931415632] Unsupported hvac_mode 'heat_cool' Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 199, in handle_call_service await hass.services.async_call( File "/usr/src/homeassistant/homeassistant/core.py", line 1820, in async_call task.result() File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in _execute_service await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)( File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 216, in handle_service await service.entity_service_call( File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 798, in entity_service_call future.result() # pop exception if have File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call await coro File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call await result File "/usr/src/homeassistant/homeassistant/components/nest/climate_sdm.py", line 287, in async_set_hvac_mode raise ValueError(f"Unsupported hvac_mode '{hvac_mode}'") ValueError: Unsupported hvac_mode 'heat_cool' 2023-05-09 19:51:25.585 WARNING (MainThread) [homeassistant.components.websocket_api.http.connection] [139724872852112] Disconnected: Did not receive auth message within 10 seconds 2023-05-09 20:00:00.203 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [139724931415632] Unsupported hvac_mode 'heat_cool' Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 199, in handle_call_service await hass.services.async_call( File "/usr/src/homeassistant/homeassistant/core.py", line 1820, in async_call task.result() File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in _execute_service await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)( File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 216, in handle_service await service.entity_service_call( File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 798, in entity_service_call future.result() # pop exception if have File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call await coro File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call await result File "/usr/src/homeassistant/homeassistant/components/nest/climate_sdm.py", line 287, in async_set_hvac_mode raise ValueError(f"Unsupported hvac_mode '{hvac_mode}'") ValueError: Unsupported hvac_mode 'heat_cool' 2023-05-09 20:30:00.223 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [139724931415632] Unsupported hvac_mode 'heat_cool' Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 199, in handle_call_service await hass.services.async_call( File "/usr/src/homeassistant/homeassistant/core.py", line 1820, in async_call task.result() File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in _execute_service await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)( File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 216, in handle_service await service.entity_service_call( File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 798, in entity_service_call future.result() # pop exception if have File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call await coro File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call await result File "/usr/src/homeassistant/homeassistant/components/nest/climate_sdm.py", line 287, in async_set_hvac_mode raise ValueError(f"Unsupported hvac_mode '{hvac_mode}'") ValueError: Unsupported hvac_mode 'heat_cool'Additional information
No response
The text was updated successfully, but these errors were encountered: