Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HA-Bridge/SmartThings Shows Bulbs Offline After Latest (26.9) Firmware Update #1107

Open
posborne opened this issue Jun 30, 2019 · 26 comments

Comments

Projects
None yet
6 participants
@posborne
Copy link

commented Jun 30, 2019

Hi, I'm a developer on the hub firwmware at SmartThings. We recently updated our local hue integration to have more direct support for "device health" within the SmartThings platform (along with other changes). As a result of these changes, some users using HA-Bridge have complained that their bulbs are now showing as offline or toggling between offline/online (see https://community.smartthings.com/t/hub-firmware-release-notes-26-9/166632/10).

I do not currently run HA-Bridge but our current suspicion is that this occurs due to HA-Bridge not fully emulating the reachable attribute of the /api/<username>/lights/<id> endpoint in some way (or not supporting that endpoint at all).

We haven't done a deep dive into this as of yet but we suspect the right fix is for HA-Bridge to provide a better emulation of the bridge. One simple solution to start out would be to "lie" and always present any attached devices as reachable.

If additional details are required from the SmartThings team just ping me on this issue and I'll do my best to get the right info back to you.

@EnGamma

This comment has been minimized.

Copy link

commented Jun 30, 2019

I am an HA-Bridge (v5.2.2), SmartThings Hub (v2), Alexa (4 Echos v1, and a few v1, v2 Echo dots) user that lost access to all my HA-Bridge devices after the ST firmware update on 6/27. I've spent much of this past weekend troubleshooting mostly to no avail. However, I have a few observations that may be useful to you. The most significant of which is that reversion to HA-Bridge v3.5.1 fixed a longstanding inability of Alexa to discover HA-Bridge devices directly (i.e., not through ST).

  1. In absence of ST=>HA-Bridge function since 6/27/2019, I tried to get Alexa to discover the HA-Bridge directly (not through ST) but have been unable to make it work. I read many comments about multiple Echo's of different versions causing conflict and scaled back to only one and ensure it was directly connected to same router as HA-Bridge on Raspberry Pi. None of this worked. No discovery of HA-Bridge devices. I varied the UPNP delay between 250ms and 1000ms without affect. I used port 80, used LINK button. Turned on UPNP for my router. Checked for IGMP issues. Troubleshooting per https://github.com/bwssytems/ha-bridge/wiki/Trouble-Shooting#nodiscover
    None of this helped.

  2. Finally, based on a clue I saw in another thread, I reverted HA-Bridge back to 3.5.1 and Alexa found all 81 HA-Bridge devices on the first try without going through ST Hub. Direct Alexa discovery of HA-Bridge devices is ultimately what I want so that the ST Hub (and the cloud) is not the loop for devices that don't require it.

  3. The loss of direct Alexa discovery is not recent. Many months ago Alexa discovered my HA-Bridge devices directly, but at some point Alexa only began discovering them through ST. For the most part this did not affect me so I ignored it. This week's debacle with the ST firmware upgraded got me into troubleshooting Alexa discovery.

  4. I still don't have ST working with HA-Bridge. That does appear to be related to the 'device health' feature. After many trials, I finally got ST to rediscover the HA-Bridge, but it was immediately declared 'OFFLINE' and hasn't changed and it never reloaded all the HA-Bridge child devices.

So what goodness in v3.5.1 should be resurrected to improve (or in my case enable) Alexa discovery? I will sit here with 3.5.1 as a stopgap but I sure would like the other enhancements you have added in subsequent revisions.

Thanks for a great piece of software.

I've attached some logs of v5.2.2 failure and v3.5.1 success in case it helps you.

discovery_success_log_v351.txt
discovery_fail_log_v522.txt

Tom

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 1, 2019

@EnGamma In the RC version of ha-bridge, I put the original upnp broadcast message from v3.5.1 in it. You can also just turn on that message from the Bridge Control tab under the use upnp original.

@posborne As far as the attributes returned on the state message of the hue to have reachable=true, I'll double check to make sure that is set as that was the default.

@EnGamma If you could list what IP's are what in your log, that would be helpful

@bwssytems bwssytems added the question label Jul 1, 2019

@danwooller

This comment has been minimized.

Copy link

commented Jul 1, 2019

I have this issue as well, smartthings doesn't find ha-bridge running RC8

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 1, 2019

@posborne Reachable in the state section is always set to true in the ha-bridge. What other state items are you looking at?

@posborne

This comment has been minimized.

Copy link
Author

commented Jul 1, 2019

@posborne Reachable in the state section is always set to true in the ha-bridge. What other state items are you looking at?

Ok, I followed up with the primary maintainer of the hue integration code and I was off base with that initial theory as confirmed by your finding.

We think the problem is probably on our end and relates to trying to use TLS for V2 bridges. Since HA-Bridge reports as a V2 bridge that was being attempted. We will likely be considering a fix short term that either uses the version of the bridge (1.24+ for TLS) as a marker for TLS support or, temporarily, just using port 80 as we have in the past. Longer term, we will of course want to use TLS whenever possible when talking to real bridges as otherwise hue credentials are in the clear.

Sorry for the confusion, I may play around with ha-bridge and get it setup on my unraid server at home so we have better dogfooding of that moving forward and test the fix we are prepping right now. The TLS switchover may be affecting some customers who have bridges on older firmware anyhow.

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 1, 2019

Great news, I may incorporate HTTPS with TLS as well as that is where the hue API is at anyway, but that will be in the future.

@EnGamma

This comment has been minimized.

Copy link

commented Jul 1, 2019

@EnGamma If you could list what IP's are what in your log, that would be helpful

.170 is ST
.155 is Raspberry Pi running HA-Bridge
.48, .49 Harmony Hubs
.23, .91 .103, .138, .147, .214 are Alexa Echos or Echo Dots

@EnGamma

This comment has been minimized.

Copy link

commented Jul 2, 2019

@EnGamma In the RC version of ha-bridge, I put the original upnp broadcast message from v3.5.1 in it. You can also just turn on that message from the Bridge Control tab under the use upnp original.

I just tried v5.3.0RC8 and it failed with 'UPNP Original (simple version)' checked. Log attached.

discovery_fail_log_v530RC8.txt

Yesterday, I tried other versions to find out where Alexa discovery stops working for me. Summary of what I have found:
5.3.0RC8 => fails
5.2.2 => fails
5.2.1 => fails
4.5.6 => fails
4.3.1 => successful Alexa discovery
3.5.1 => successful Alexa discovery

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 2, 2019

@EnGamma This is very weird as your echos do ask for the hue lights from the ha-bridge which tells them they found lights... The message below is one of them:

Jul 1 19:18:58 HA-PI java[29458]: 2019-07-01 19:18:58,344 [qtp9662400-92] INFO com.bwssystems.HABridge.hue.HueMulator - Traceupnp: hue lights list requested by user: 6GtNFc2ewsQuUlGtH4NNM5hmTgiisr5Jh4iafloa from address: 192.168.124.103

If you see these messages, the echo has discovered the ha-bridge and is asking for what lights it has.

Could you humor me and only have one echo running and the ha-bridge on RC8. Also, please uncheck upnp original when you do this.

Also, I determined that the difference in v3.5.1 and the newer version is not what I put into the latest RC versions. I have that covered in RC9, but it is ever so slight.

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 2, 2019

@EnGamma The new release RC9 is out and now emulates the v3.5.1 properly when upnp original is selected. Also, this release has mDNS capabilities.

@EnGamma

This comment has been minimized.

Copy link

commented Jul 3, 2019

Could you humor me and only have one echo running and the ha-bridge on RC8. Also, please uncheck upnp original when you do this.

I've done this with both RC9 and then RC8, checked and unchecked and all failed. Log attached.

discovery_log_v530RC9_v530RC8.txt

@EnGamma

This comment has been minimized.

Copy link

commented Jul 3, 2019

After failing with RC8 and RC9 I've reverted back to v4.3.1 for now. Here is a clean log showing successful Alexa Discovery with this version. Works 100% of the time.

discovery_success_log_v431.txt

@EnGamma

This comment has been minimized.

Copy link

commented Jul 4, 2019

https://community.smartthings.com/t/hue-integration-stopped-working-27-june-2019/166962/51

Yes, this addresses the 6/27 problem with SmartThings, but it does not address the direct Alexa (Echo)=>HA-Bridge discovery problem, which predates that issue. The 6/27 ST problem, however, is what got me back into troubleshooting why the direct Alexa discovery of HA-Bridge devices was failing.

For me, and I would imagine numerous others, not requiring ST to be in the middle is an important benefit. I have lost control before due to ST issues, so one less dependency is desirable.

@EnGamma

This comment has been minimized.

Copy link

commented Jul 4, 2019

@bwssytems: Since I have a repeatable HA-Bridge version-dependent issue, I am willing to support any further troubleshooting I can help you with.

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 5, 2019

Thanks, still trying to figure out what's different as the code I see send to reflect what the older version is now doing. Will review some more. Also, don't use RC8 as that was not correct for the discovery portion.

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 9, 2019

@EnGamma From your RC9 log, the echo asks for the lights list, but seems to do it multiple times with this message: Jul 2 18:49:41 HA-PI java[20294]: 2019-07-02 18:49:41,171 [qtp31486706-93] INFO com.bwssystems.HABridge.hue.HueMulator - Traceupnp: hue lights list requested by user: 6GtNFc2ewsQuUlGtH4NNM5hmTgiisr5Jh4iafloa from address: 192.168.124.103

So, the echo is fine with the UPNP handshake, but it seems it does not like something in the lights list. Would it be possible for you to post your device.db?

@EnGamma

This comment has been minimized.

Copy link

commented Jul 11, 2019

@EnGamma From your RC9 log, the echo asks for the lights list, but seems to do it multiple times with this message: Jul 2 18:49:41 HA-PI java[20294]: 2019-07-02 18:49:41,171 [qtp31486706-93] INFO com.bwssystems.HABridge.hue.HueMulator - Traceupnp: hue lights list requested by user: 6GtNFc2ewsQuUlGtH4NNM5hmTgiisr5Jh4iafloa from address: 192.168.124.103

So, the echo is fine with the UPNP handshake, but it seems it does not like something in the lights list. Would it be possible for you to post your device.db?

@cbrittler

This comment has been minimized.

Copy link

commented Jul 11, 2019

Does anyone know if the SmartThings firmware will be updated to support the HA Bridge? I’m still unable to connect and show HA Bridge offline.

@audiofreak9

This comment has been minimized.

Copy link
Collaborator

commented Jul 11, 2019

I installed the smartapp Hue_B_Smart and got mine back.

Smartthings made a change from port 80 to port 443 for discovery which stopped Gen 1 Hue and HA-Bridge Smartthings connections.

There is talk of an update, but that was said to be released in the next several weeks.

@EnGamma

This comment has been minimized.

Copy link

commented Jul 12, 2019

@audiofreak9

This comment has been minimized.

Copy link
Collaborator

commented Jul 12, 2019

Meanwhile support will manually reset the port to 80 for you. This is not
normally user settable. It may also revert to port 443 under certain
circumstances, I believe.

That is true, support did change my HA-Bridge in ST to port 80, however I was unable to do a device discovery since that is performed on port 443 now with the ST update.

My lesson learned, don't delete devices in ST until there is an update to ST. I deleted all my devices and the bridge from ST trying to troubleshoot before I knew there was a ST update.

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 12, 2019

I have added https support into the latest branch (but not released it yet), but it will require you to build tls keys to do this. Instructions need to be written.

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 12, 2019

@EnGamma One suggestion maybe encoding I changed from one version to the other for the return of the lights list. Could you try changing the two entries that have an apostrophe in it "Gregory's" and Kathy's" to be without it, "Gregorys" and Kathys", as the Alexa does not care about those.

<Edit>
Also, I would suggest doing a renumber under RC9 as that will make the ids and unique ids cleaner.

@EnGamma

This comment has been minimized.

Copy link

commented Jul 13, 2019

One suggestion maybe encoding I changed from one version to the other for the return of the lights list. Could you try changing the two entries that have an apostrophe in it "Gregory's" and Kathy's" to be without it, "Gregorys" and Kathys", as the Alexa does not care about those.

<Edit>
Also, I would suggest doing a renumber under RC9 as that will make the ids and unique ids cleaner.

@bwssytems

  1. Removing apostrophes had no effect.
  2. I renumbered under RC9 (to simpler 3-digit numbers) and Alexa is now discovering all devices!

Thank you. I will continue to monitor.

@bwssytems

This comment has been minimized.

Copy link
Owner

commented Jul 15, 2019

@EnGamma Great! It looks like you have been using ha-bridge from the beginning as you had the high order numbers that were replaced. And the numbering scheme is now settable from a given start point. Also when renumbering, the Hue Unique Ids are regenerated to be more correct in the latest version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.