Emulated Hue Devices 'unresponsive' in Alexa

[guardianbs]

The problem

I don't know what the area of responsibility is for this one but here's the problem.

I have 17 entities exposed as emulated hue devices. These have been configured for some time and have been working perfectly with Alexa able to control them via routines.

A few of days ago these stopped working consistently. Looking in the Alexa app and the status of these devices are now fluctuating between ok (i.e 'on' / 'off'), 'New Device' and 'Device is unresponsive'. Alexa routines cannot set these entities when 'unresponsive'.

Home Assistant shows the entities normally, i.e. as available. I have tried the following but the problem remains

Home Assistant core 2022.6.6, 2022.6.7, 2022.7.1. 2022.7.2
Rebooted all Echo devices (8 in total)
Rebooted internet router
Deleted device in Alexa app and re-discovered
Turned on emulated_hue logging - no errors showing.

Any ideas ? Is this purely an Alexa problem or some change in the Alexa / Emulated_hue interface ?

What version of Home Assistant Core has the issue?

core 2022.6.6 / 2022.6.7 / 2022.7.1 / 2022.7.2

What was the last working version of Home Assistant Core?

core 2022.6.6 (previously)

What type of installation are you running?

Home Assistant OS

Integration causing the issue

emulated_hue

Link to integration documentation on our website

https://www.home-assistant.io/integrations/emulated_hue/

Diagnostics information

emulated_hue_log.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[probot-home-assistant]

emulated_hue documentation
emulated_hue source
_{^{(message by IssueLinks)}}

[probot-home-assistant]

Hey there @bdraco, mind taking a look at this issue as it has been labeled with an integration (emulated_hue) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[bdraco]

What version of Home Assistant Core has the issue?

core 2022.6.6 / 2022.6.7 / 2022.7.1 / 2022.7.2

What was the last working version of Home Assistant Core?

core 2022.6.6 (previously)

Your versions imply something was broken in 2022.6.7.

It is unclear which versions were working and which ones are not working from your message.

[guardianbs]

Sorry for the confusion. I know it was working correctly for a long time on core 2022.6.7. I then updated to 2022.7.1 and then 2022.7.2. Shortly after updating to 2022.7.2 I noticed the problem described so rolled back to 2022.6.7, but there was still the problem, so I rolled back to 2022.6.6 which I’m currently on, but there is still the problem. So it would appear that the problem is independent of the HA core version, suggesting that maybe something has changed in the Alexa / HA interface. From: J. Nick Koston Sent: Tuesday, July 12, 2022 1:30 PM To: home-assistant/core Cc: guardianbs ; Author Subject: Re: [home-assistant/core] Emulated Hue Devices 'unresponsive' in Alexa (Issue #75057) What version of Home Assistant Core has the issue? core 2022.6.6 / 2022.6.7 / 2022.7.1 / 2022.7.2 What was the last working version of Home Assistant Core? core 2022.6.6 (previously) Your versions imply something was broken in 2022.6.7. It is unclear which versions were working and which ones are not working from your message. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[bdraco]

Can you give the instructions in this PR a shot?

#39539

[guardianbs]

Hi, Thank you for the response. If I’m reading the PR correctly it suggests that only entities defined as non dimmable would be having the 'unresponsive' problem but the 5 entities I have defined as dimmable are also experiencing the 'unresponsive' problem. From: J. Nick Koston Sent: Tuesday, July 12, 2022 3:50 PM To: home-assistant/core Cc: guardianbs ; Author Subject: Re: [home-assistant/core] Emulated Hue Devices 'unresponsive' in Alexa (Issue #75057) Can you give the instructions in this PR a shot? #39539 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[bdraco]

Ignore the dimmable vs non dimmable for the purposes here as the reset instructions provided in that PR are applicable as they usually resolve these type of issues.

Emulated hue does not track entities by unique id and instead by entity id. Any change in the entity id can cause things to get in a bad state so this is a good step to make sure that's not the problem.

At some point the integration needs to be migrated to use the unique ids for tracking entities but that would require everyone to reset their configs so it hasn't been done yet

[Cavekeeper]

I can confirm that. I was able to use the Emulated HUE in my Busch Jaeger free@home home control system for a long time. Now it doesn't work anymore. So I researched back and tried the following.
With Home Assistant Core 2022.6.6, the Emulated HUE is visible in the home control and can be activated.
With Home Assistant 2022.7.5, the Emulated HUE is no longer visible. It doesn't work.

[bdraco]

Is there anything in the logs?

Did you try the reset procedure I linked above ? (Probably not relevant in your case)

Can you try turning on debug logging?

[bdraco]

Also if you can get a packet dump pcap with the differences between 2022.7.x and 2022.6.x with your devices we can likely work out what needs to be adjusted

Unfortunately all my devices produce functionally identical json between these versions so it's not something I can do with mine

[Cavekeeper]

I installed the last update 2022.7.6 on my dev system and then activated the Emulated HUE Bridge again. I checked the log from the Busch Jaeger free@home home control system and found the following entry.

Sat Jul 23 16:27:35 2022 mrha_huecontroller | EE hue_bridge.cpp 358 Have a network reply timeout http://192.168.178.75:8300/api/nouser/lights
Sat Jul 23 16:27:39 2022 mrha_huecontroller | EE hue_bridge.cpp 358 Have a network reply timeout http://192.168.178.75:8300/api/nouser/lights
Sat Jul 23 16:27:43 2022 mrha_huecontroller | EE hue_bridge.cpp 358 Have a network reply timeout http://192.168.178.75:8300/api/nouser/lights
Sat Jul 23 16:27:47 2022 mrha_huecontroller | EE hue_bridge.cpp 358 Have a network reply timeout http://192.168.178.75:8300/api/nouser/lights

[bdraco]

Are you using a container with python 3.10?

[Cavekeeper]

Yes, it seems like.

Version	core-2022.7.6
Installationstyp	Home Assistant OS
Entwicklung	false
Supervisor	true
Docker	true
Benutzer	root
Virtuelle Umgebung	false
Python-Version	3.10.5
Betriebssystemfamilie	Linux
Betriebssystem-Version	5.15.52
CPU-Architektur	x86_64
Zeitzone	Europe/Berlin
Configuration Directory	/config

[Cavekeeper]

I now had the opportunity to test it with version 2022.7.7 and OS 8.4. Unfortunately without success. But it is interesting that I can pair the Emulated HUE and also see the associated entities. Unfortunately, these cannot be switched. Neither from HA nor from the Smart Home device in which the Emulated HUE was paired. That actually means that the connection is established and that there is basically communication. But why can't a switching action be executed?

[bdraco]

Can you capture the network traffic between the device and home assistant when you try to switch ?

[Cavekeeper]

Yes, this is a good idea. I have checked the traffic between through Wireshark (ip host 10.xx.xx.xx and ip host 10.yy.yy.yy).
I tried to record the data traffic between the working variant 2022/6/7 and the smart device. But somehow I get no traffic displayed. Wireshark actually works. I might be missing something else.

[Cavekeeper]

Hello @bdraco Since I have a managed switch, I was able to mirror the ports and make a recording with the working version 2022-6-7 and the problematic version 2022-7-7. I have the recordings here for download. Maybe you can glean something from the Wireshark recordings. I pressed the Emulated HUE switches several times during the recording. Comments are stored in the Wireshark recording.

[bdraco]

The dumps helped. The problem is that in 2022.7.7+ the data has the Content-Encoding: gzip header but its not gziped so the compression is not working for some reason.

Here is a screenshot

[bdraco]

I haven't upgraded my production instance that uses emulated_hue to python 3.10 yet so that likely why I'm not seeing the issue

% nc 192.168.107.5 8300
GET /api/nouser/lights HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: 192.168.178.75:8300

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 245
Content-Encoding: gzip
Date: Tue, 02 Aug 2022 00:29:10 GMT
Server: Python/3.9 aiohttp/3.8.1

??=O?0???us*%v!?mE
                  څ??sri,?6?U???ĀX?Nw??}?
                                         t?????;?27?????_
                                                         ??2??-SN?R2?n@([um???ȁb? x{s?S,Tv?%?˹mqC??Qk???_IY8K.O4?8|?ەD??є?.&??9Ě??IU6???????ZY,5>:/????x?li)O???+w??{s6?$????ߍU????????I????O??{?

[probot-home-assistant]

http documentation
http source
_{^{(message by IssueLinks)}}

[probot-home-assistant]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (http) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[bdraco]

I upgraded to python 3.10 and unfortunately it didn't break

% nc 192.168.107.5 8300
GET /api/nouser/lights HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: 192.168.178.75:8300

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 245
Content-Encoding: gzip
Date: Tue, 02 Aug 2022 02:59:57 GMT
Server: Python/3.10 aiohttp/3.8.1

??=O?0???us*%v!?mE
                  څ??sri,?6?U???ĀX?Nw??}?
                                         t?????;?27?????_
                                                         ??2??-SN?R2?n@([um???ȁb? x{s?S,Tv?%?˹mqC??Qk???_IY8K.O4?8|?ەD??є?.&??9Ě??IU6???????ZY,5>:/????x?li)O???+w??{s6?$????ߍU????????I????O??{?

[bdraco]

Which hardware platform are you using?

[Cavekeeper]

The production system runs on an RPI3 and the dev system on a VM linux server.

[bdraco]

If you manually connect, and send the http header as shown above, do you see it coming over the wire uncompressed?

[Cavekeeper]

Unfortunately there was probably a misunderstanding. My hardware (working) platform is a Mac. That's why I also have the problem to run the above command. I get the error "zsh: command not found: GET". Is there another way to send the command line manually?

[bdraco]

nc should be installed on macos. If not you can install it with homebrew

[Cavekeeper]

I could not connect to port 8300 with netcap. But it worked with port 80. So I configured Emulated HUE on port 80. I did that for both systems 2022.6.7 and the new 2022.8.1. The results are the same in both cases.

2022.6.7

`~ nc 192.168.178.3 80
GET /api/nouser/lights HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: 192.168.178.5:80

HTTP/1.1 404 Not Found
Server:
Date: Fri, 05 Aug 2022 09:45:23 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 639
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5bb16380-27f"
Content-Encoding: gzip

??N?)?I?5_??{??{?.??O???-???ۻ??M
??
J????????LFJo> ?sML?n??#mJ?|???L?h???d~????Z*?=???f?ABP,?3??#?
?7kt
>????B+??E?C???ˈý??/??bƢ?6n]?=?N?|e_W?F???ޘAi+?Q<?]T??? ?O?????
?}ER???v?.?XhÜ?????=b?? ?[x????c?j
????Jo?
?????d?[|Nޔ???????k????a?
U?0n? ?̔B??NpW?0?&P?(+?{??\lAp?s#??ȑ?$????gϵs??F{m?Xn??8?BΊui?ģ?q8?^?瘾/???m?髿?1)J??D???2B?|!E???b?Ejh?.?C??<??~??;?QJى8ԫ3???8?fJ??mN?????7=??-????O??J??I2?
`

2022.8.1

`~ nc 192.168.178.3 80
GET /api/nouser/lights HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: 192.168.178.75:80

HTTP/1.1 404 Not Found
Server:
Date: Fri, 05 Aug 2022 10:07:11 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 639
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5bb16380-27f"
Content-Encoding: gzip

??N?)?I?5_??{??{?.??O???-???ۻ??M
??
J????????LFJo> ?sML?n??#mJ?|???L?h???d~????Z*?=???f?ABP,?3??#?
?7kt
>????B+??E?C???ˈý??/??bƢ?6n]?=?N?|e_W?F???ޘAi+?Q<?]T??? ?O?????
?}ER???v?.?XhÜ?????=b?? ?[x????c?j
????Jo?
?????d?[|Nޔ???????k????a?
U?0n? ?̔B??NpW?0?&P?(+?{??\lAp?s#??ȑ?$????gϵs??F{m?Xn??8?BΊui?ģ?q8?^?瘾/???m?髿?1)?2B?|!E???b?Ejh?.?C??<??~??;?QJى8ԫ3???8?fJ??mN?????7=??-????O??J??I2?`

It is interesting that the pairing also works with 2022.8.1. The emulated HUE device from HA can also be seen in the smart home center. Then I can do the following with it:
In the smart home center: I turn the Emulate HUE switch ON = the switch in the smart home center goes ON and immediately OFF again. In HA, the switch goes ON and stays ON.
In HA: I toggle the Emulated HUE switch ON and OFF = it doesn't happen in the smart home center.

It looks like there is only one-way communication.
As I mentioned before, this problem only occurs from 2022.7.x onwards. Everything worked fine with version 2022.6.7.
I also have a log file from the Smart Home Center which you can download here.

[bdraco]

Is your emulated hue running on 80? You shouldn't be getting a 404 response. I'm not sure you're testing the emulated hue webserver, but instead checking a different Web server on the same IP

[Cavekeeper]

Sorry my mistake. Incorrect IP address. Ok, now the right one again:

2022.7.6

`% nc 192.168.178.5 8300
GET /api/nouser/lights HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: 192.168.178.3:8300

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 294
Content-Encoding: gzip
Date: Sat, 06 Aug 2022 11:07:02 GMT
Server: Python/3.9 aiohttp/3.8.1

쑅<??l??5?HQr?HYWօ?J??$e?M?=yvgv?3g.?dg"D???Q?0??v????&7P?G7!??&?:?̙?0-Tk?O
??<???M???!?|???
?o?^?墬?O?E???[????q??/?~???|^?i??*?⧼N4??y? ?Q????@䘨(??/?JaO???t???
??Ⱥ?9??n%$?)j???
es?????@?M???-???Y`

2022.8.1

`% nc 192.168.178.75 8300
GET /api/nouser/lights HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: 192.168.178.3:8300

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 209
Content-Encoding: gzip
Date: Sat, 06 Aug 2022 11:11:12 GMT
Server: Python/3.10 aiohttp/3.8.1

]??n?0
???젱??(7o^?
)?36U
?OQ)???^??ԍ???;ޡ?C?C
j???????#??3??+??I8^/?r?B1?HI??f ???Ӳ?ũ.????3l?3??????W㝳d]q©????
?L??q?5??U??Z?Sx:?8??U??Zy??g?/??y:????}?4?G`

[Cavekeeper]

You wrote above that GZIP compression could be a possible problem. In my last attempt to address the Emulated HUE server directly, the GZIP compression obviously worked. Is it perhaps due to the Python 3.10 version? I would like to update to HA to get the new features. But unfortunately, the emulated HUE problem prevents me from doing so because I need this function in my smart home controller. Is there still a chance to clean it up somehow?

[bdraco]

Right now I don't have any idea how to trigger the issue. Your pcap shows the issue but unless we figure out how to trigger it manually there isn't much we can do to fix it.

[ktown007]

I also have the 'unresponsive' in Alexa only for the zwave devices. It happened after the upgrade to zwave-js. I can see the devices in the REST GET(/api/v2/lights) and change state via PUT(api/v2/lights/switch.r5/state). If I delete one from app Alexa does not rediscover them. I can see errors(Entity not found) with some of the old pre zwave-js entity names. I did delete all devices on web and Alexa rediscovered the zigbee/tasmota lights and switches. Even though I deleted them some old data is still cached, maybe the unique id's do not match new entity id's. Does "config/emulated_hue_ids.json" still need to be deleted I do not see this file?

I deleted devices, and did some pcaps and the alexa is not (re)discovering the devices. Some of the old entity names worked but now I can not get them back.

[Ramias1]

2022.8.7 core running in docker with its own IP; publishing Hue on port 80.

New user of the Emulated Hue Bridge here. Removed all of my echo devices, deleted all of my echo smart home devices, added back one echo device and discovered devices (all devices were on - i.e. fan switches turned on, lights turned on, TV's turned on during Alexa discovery). They are all discovered but real lights show correctly as on/off light switches. Other switches show as dimmable lights. I have a zwave switch (Defined through a helper as a Light and published to Emulated Hue as a light). It works fine. My switches (which show as dimmable lights in Alexa) sometimes work but I get a "not responding" at other times.

[guardianbs]

Hmm. Not looking forward to deleting and discovering 108 devices and then updating the 55 routines that use the devices, not to mention deleting and re-registering 8 echo devices . . . Oh well, got to done I suppose. Just hope it works . . . From: J. Nick Koston Sent: Tuesday, July 12, 2022 5:40 PM To: home-assistant/core Cc: guardianbs ; Author Subject: Re: [home-assistant/core] Emulated Hue Devices 'unresponsive' in Alexa (Issue #75057) Ignore the dimmable vs non dimmable for the purposes here as the reset instructions provided in that PR are applicable as they usually resolve these type of issues. Emulated hue does not track entities by unique id and instead by entity id, any change in the entity id name can cause things to get in a bad state so this is a good step to make sure that's not the problem. At some point the integration needs to be migrated to use the unique ids for tracking entities but that would require everyone to reset their configs so it hasn't been done yet — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[derjoerg]

Any news?

[ktown007]

Alexa Discovery of Emulated Hue devices is still broken for me.

[cncb-gh]

Me too. After my recent update to Home Assistant 2022.12.9 (Docker) a few days ago, they all now show "Device is unreponsive" in the Alexa app. I tried to delete one, but it does not discover it again.

[ltpitt]

For me it discovers successfully but it forgets discovered devices in hours / days.