No connection with Eneco Toon (Quby) #23

kurniawan77 · 2017-07-17T12:44:12Z

Hi,

I have got your diyHue emulator working fine with the official phillips hue apps together with a wemos d1 mini and a neopixel ring. Thanks a lot!

I also have a Quby (dutch Toon Eneco) which serves as a thermostat, zwave and Hue controller but it simply could'nt find the Hue bridge emulator. It seems that the Toon listens to others broadcasts then your simulator does.

It would be great to have your simulator working with the Toon. I have found some info over here: https://github.com/ewjmulder/hue-bridge-simulator/issues and especially here: https://raw.githubusercontent.com/ewjmulder/program-your-home/master/eneco-toon/IDEAS.txt which explains the broadcast of this emulator.

I am not a dev but i certainly would love to help with testing if you would have a dive in this.

regards kurniawan77

kurniawan77 · 2017-07-17T12:50:38Z

UPnP response(s) from Hue: -> Note: aren't these just broadcasts? -> No, these are responses!

HTTP/1.1 200 OK
CACHE-CONTROL: max-age=100
EXT:
LOCATION: http://192.168.2.100:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
ST: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b::upnp:rootdevice

HTTP/1.1 200 OK
CACHE-CONTROL: max-age=100
EXT:
LOCATION: http://192.168.2.100:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
ST: uuid:2f402f80-da50-11e1-9b23-00178818572b
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

HTTP/1.1 200 OK
CACHE-CONTROL: max-age=100
EXT:
LOCATION: http://192.168.2.100:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
ST: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

HTTP description.xml response:

<?xml version="1.0" encoding="UTF-8" ?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<URLBase>http://192.168.2.102:80/</URLBase>
<device>
<deviceType>urn:schemas-upnp-org:device:Basic:1</deviceType>
<friendlyName>Philips hue (192.168.2.102)</friendlyName>
<manufacturer>Royal Philips Electronics</manufacturer>
<manufacturerURL>http://www.philips.com</manufacturerURL>
<modelDescription>Philips hue Personal Wireless Lighting</modelDescription>
<modelName>Philips hue bridge 2012</modelName>
<modelNumber>929000226503</modelNumber>
<modelURL>http://www.meethue.com</modelURL>
<serialNumber>00178818572b</serialNumber>
<UDN>uuid:2f402f80-da50-11e1-9b23-00178818572b</UDN>
<presentationURL>index.html</presentationURL>
<iconList>
<icon>
<mimetype>image/png</mimetype>
<height>48</height>
<width>48</width>
<depth>24</depth>
<url>hue_logo_0.png</url>
</icon>
<icon>
<mimetype>image/png</mimetype>
<height>120</height>
<width>120</width>
<depth>24</depth>
<url>hue_logo_3.png</url>
</icon>
</iconList>
</device>
</root>

These are broadcasts, groups of 6 (3 doubles of different UPnP devices), with an interval of approx. 1 minute:

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b::upnp:rootdevice

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b::upnp:rootdevice

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: uuid:2f402f80-da50-11e1-9b23-00178818572b
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: uuid:2f402f80-da50-11e1-9b23-00178818572b
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

When simulating the above, Toon will actually pick up our Hue Bridge simulation!!
It gets the description.xml and displays the MAC address,
which it gets from the tag in the description.xml (so not the real one from the network card)

mariusmotea · 2017-07-17T14:18:26Z

Hi,

Will be really hard for me to fix this issue without the device, but i can provide you the needed information in order to fix it. The registration process is made from 2 steps, one is ssdp discovery of the hue bridge, second one is the http POST message to receive the username from hue bridge, but from what i understand we don't go to second step. This can be saw if you execute HueEmulator.py manually in shell to see entire output. SSDP reply is this line in the code: Response_message = 'HTTP/1.1 200 OK\r\nHOST: 239.255.255.250:1900\r\nEXT:CACHE-CONTROL: max-age=100\r\nLOCATION: http://' + get_ip_address() + ':80/description.xml\r\nSERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.16.0\r\nhue-bridgeid: ' + mac.upper() + '\r\nST: urn:schemas-upnp-org:device:basic:1\r\nUSN: uuid:2f402f80-da50-11e1-9b23-' + mac. Here \r\n means new line, + mac append mac address without ":", + get_ip_address() + append ip address. If something is wrong here you can manually edit to fit Toon requirements ( i can see NT: urn:schemas-upnp-org:device:basic:1 is missing). If you find a working response please let me know, maybe you will create a pull request with your changes.

mariusmotea · 2017-07-18T07:30:23Z

Hi again. Today i spent a few minutes to check again what can be the issue with SSDP message response and i saw there is a small difference from original hue bridge ssdp reply found on
hue forum, more exactly hue-bridgeid line contain "FFFE" string in the middle of mac address. To fix this replace line 69 with the following:
Response_message = 'HTTP/1.1 200 OK\r\nHOST: 239.255.255.250:1900\r\nEXT:CACHE-CONTROL: max-age=100\r\nLOCATION: http://' + get_ip_address() + ':80/description.xml\r\nSERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.16.0\r\nhue-bridgeid: ' + (mac[:6] + 'FFFE' + mac[6:]).upper() + '\r\nST: urn:schemas-upnp-org:device:basic:1\r\nUSN: uuid:2f402f80-da50-11e1-9b23-' + mac

If still don't work, then you must setup the mac address manually on you raspberry to start with "00:17:88:xx:xx:xx". This is the prefix used by all Hue Bridge devices.

kurniawan77 · 2017-07-19T06:22:07Z

Hi mariusmotea.

Thank you for your reply, diving in and your time. I have replaced the line but unfortunately it still didn't work with the Eneco Toon. Now what do you mean by manually setup the mac address. Where do i do this?
But...
From what i can read on https://github.com/ewjmulder/hue-bridge-simulator

###UPnP traffic###

The simulated bridge mimics the UPnP behavior from the Philips Hue bridge as closely as possible. This means: every minute 6 NOTIFY messages will be sent. Three different messages, each one sent twice. This behavior is based on sniffing the local network messages of a real Philips Hue bridge. The messages contain information about the Hue bridge device. Their content is exactly identical to the ones from an actual Philips Hue bridge, except for the LOCATION: and uuid: parts. See the HueBridgeUpnpSimulator class for more information. These NOTIFY messages are enough for Toon to pick up this simulated bridge, so no further UPnP protocol functionality is implemented. This also means the simulated bridge will not respond to UPnP M-SEARCH queries. This might be added in the future.

The simulated bridge will also serve the description.xml file that is referred to in the UPnP NOTIFY messages. The contents of this description.xml file does exactly match the one from the real Philips Hue bridge, except for the host, port and MAC address. Also the friendly name is appended with 'simulation'.

So i don't expect it to be in the MAC address. But i could be wrong...

contents of the description.xml

<?xml version="1.0" encoding="UTF-8" ?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<URLBase>http://[HOST]:[PORT]/</URLBase>
<device>
<deviceType>urn:schemas-upnp-org:device:Basic:1</deviceType>
<friendlyName>Philips hue simulator</friendlyName>
<manufacturer>Royal Philips Electronics</manufacturer>
<manufacturerURL>http://www.philips.com</manufacturerURL>
<modelDescription>Philips hue Personal Wireless Lighting</modelDescription>
<modelName>Philips hue bridge 2012</modelName>
<modelNumber>929000226503</modelNumber>
<modelURL>http://www.meethue.com</modelURL>
<serialNumber>[MAC]</serialNumber>
<UDN>uuid:2f402f80-da50-11e1-9b23-[MAC]</UDN>
<presentationURL>index.html</presentationURL>
<iconList>
<icon>
<mimetype>image/png</mimetype>
<height>48</height>
<width>48</width>
<depth>24</depth>
<url>hue_logo_0.png</url>
</icon>
<icon>
<mimetype>image/png</mimetype>
<height>120</height>
<width>120</width>
<depth>24</depth>
<url>hue_logo_3.png</url>
</icon>
</iconList>
</device>
</root>

mariusmotea · 2017-07-19T07:01:30Z

Temporary change of mac address can be done with command:
ifconfig eth0 down hw ether 00:17:88:01:23:45 && ifconfig eth0 up
Restart HueEmulator to get the new mac address. Please note that you will lose ssh access during interface restart. Regarding what you pasted there, there are some differences that i saw. My hue emulator reply to M-SEARCH queries and in rest it will not send any NOTIFY messenges. However i saw all hue applications try to detect the hue bridge with an M-SEARCH queries, i suspect Toon do the same. I propose to clarify some aspects. Please start HueEmulator.py in shell, then try to connect Toon to it and paste here entire output. A successful connection will log these steps:

"Sending M Search response"
GET /description.xml
POST /api
GET /api/received_user_id/config

I need to know exactly on what step it fail. Please don't open Hue Application because will populate the log with big number of requests that are not important for us.

kurniawan77 · 2017-07-23T10:54:41Z

Hi there,

I have changed line 69 as above and changed the mac address of the pi as above. Restarted HueEmulator in shell and all i get is:

pi@raspberrypi ~/diyHue/BridgeEmulator $ sudo ./HueEmulator.py
config loaded
lights address loaded
starting ssdp...
 Starting httpd...

Even when pressing search on the Toon has no effect. After this i opened up the Hue App on my android and voila... it connects to the hue app. But no way i get a response from Toon...
So it is definitely not a MAC address thing i guess

mariusmotea · 2017-07-23T19:00:22Z

Ok, so we are stuck on device discovery. Afraid you need some tools to sniff ssdp data, i recommend this: https://github.com/lianwutech/ssdp . Bridge emulator will reply only to M-SEARCH messages that contain "ssdp:all", so i possible that there are some extra space in the request from Toon. I read the links that you provide, did you try to reboot Toon? It seems that only after reboot it perform an M-SEARCH

Let's reboot are thoughts and also Toon and see what happens then
First it NBNS itself on the network as ENECO-001 and TOON
Then it actually does do a active UPnP M-SEARCH on the network!
A small time later our dear Hue bridge responds with it's known UPnP reply (see below).
Not much later Toon HTTP gets the description from Hue and now knows it has found a bridge!```

mariusmotea · 2017-07-25T07:51:31Z

Hi,

Yesterday i made some tests and it seams official Hue app will not pair with the new change. Please revert (mac[:6] + 'FFFE' + mac[6:]).upper() to mac.upper()

kurniawan77 · 2017-07-25T11:10:04Z

Hi there,

I was also trying out some stuff. The above (mac[:6] + 'FFFE' + mac[6:]).upper() has no bad effects for me. All Hue apps connect except for the Toon. But I will revert and test.

I have tried with a couple of reboots with the Toon but no hue bridge can be found. All i can see with wireshark that it indeed NBBS itself a few times after reboot but i dont see any M-SEARCH

"14553","376.240178","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<03>"
"14554","376.240179","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<00>"
"14555","376.241805","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<20>"
"14556","376.241806","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<03>"
"14557","376.252613","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<00>"
"14558","376.252614","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<00>"
"14559","376.254160","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<1e>"
"14591","377.255355","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<20>"
"14592","377.255356","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<03>"
"14593","377.256134","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<00>"
"14594","377.257748","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<20>"
"14595","377.257749","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<03>"
"14596","377.257889","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<00>"
"14597","377.257889","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<00>"
"14599","377.264082","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<1e>"
"14627","378.252579","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<20>"
"14628","378.252581","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<03>"
"14629","378.253144","192.168.129.111","192.168.129.255","NBNS","110","Registration NB ENECO-001-06791<00>"
"14630","378.253895","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<20>"
"14631","378.291345","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<03>"
"14632","378.291347","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<00>"
"14637","378.294975","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<00>"
"14638","378.296305","192.168.129.111","192.168.129.255","NBNS","110","Registration NB TOON<1e>"
"16709","446.598556","192.168.129.111","192.168.129.255","BROWSER","266","Host Announcement ENECO-001-067913, Workstation, Server, Print Queue Server, Xenix Server, NT Workstation, NT Server"
"16710","446.598681","192.168.129.111","192.168.129.255","BROWSER","266","Host Announcement TOON, Workstation, Server, Print Queue Server, Xenix Server, NT Workstation, NT Server"
"20632","566.708819","192.168.129.111","192.168.129.255","BROWSER","266","Host Announcement ENECO-001-067913, Workstation, Server, Print Queue Server, Xenix Server, NT Workstation, NT Server"
"20633","566.708997","192.168.129.111","192.168.129.255","BROWSER","266","Host Announcement TOON, Workstation, Server, Print Queue Server, Xenix Server, NT Workstation, NT Server"
"24402","686.818420","192.168.129.111","192.168.129.255","NBNS","92","Name query NB TOON<1d>"
"24466","688.832509","192.168.129.111","192.168.129.255","NBNS","92","Name query NB TOON<1d>"
"24498","689.834556","192.168.129.111","192.168.129.255","NBNS","92","Name query NB TOON<1d>"
"24524","690.836128","192.168.129.111","192.168.129.255","NBNS","92","Name query NB TOON<1d>"
"24548","691.838539","192.168.129.111","192.168.129.255","BROWSER","225","Browser Election Request"

Maybe if ewjmulder will help us... i'll try to contact him

mariusmotea · 2017-07-25T12:22:39Z

Found something that may help : https://github.com/sagen/hue-upnp. See if Toon is able to detect this. Also to sniff ssdp you can use this: https://github.com/ganeshpr/ssdp. Script ssdp.py don't have any execution points, so just append at the end of the script ssdp_search("nothing"). If you will perform a detection with Philips Hue application you will able to see this:

➜  ssdp git:(master) ✗ python ssdp.py

 waiting to recieve
received 90 bytes from ('192.168.0.175', 41093)
M-SEARCH * HTTP/1.1
HOST:239.255.255.250:1900
ST:ssdp:all
Man:"ssdp:discover"
MX:3

kurniawan77 · 2017-07-25T16:11:22Z

Hmmmm... nothing on the Toon while executing hueUpnp.py and after exit i get:

pi@raspberrypi ~/hue-upnp-master $ python2.7 hueUpnp.py
^CExiting
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "hueUpnp.py", line 146, in run
    self.server.serve_forever()
  File "/usr/lib/python2.7/SocketServer.py", line 236, in serve_forever
    poll_interval)
  File "/usr/lib/python2.7/SocketServer.py", line 155, in _eintr_retry
    return func(*args)
  File "/usr/lib/python2.7/SocketServer.py", line 455, in fileno
    return self.socket.fileno()
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
  File "/usr/lib/python2.7/socket.py", line 170, in _dummy
    raise error(EBADF, 'Bad file descriptor')
error: [Errno 9] Bad file descriptor

kurniawan77 · 2017-07-26T14:13:37Z

I can see broadcasts from hueUpnp.py script with wireshark:

But when running your script i don't get any broadcasts.

ewjmulder has responded to my question: https://github.com/ewjmulder/hue-bridge-simulator/issues/2#issuecomment-318017441

I'll try to help you with this. BTW, funny to see you guys quoting my personal 'braindump' in IDEAS.txt :-D
I think that contains the key to the solution: Toon actually seems not to use the M-SEARCH for discovering the Hue bridge, it only uses the 6 'broadcast' messages that the bridge does automatically every minute. As stated in the documentation, I just had to exactly replicate these messages for Toon to find my simulated bridge. I only changed the location of the description.xml, the mac address and the bridge name, as you can see in my code.

Did you try to auto-broadcast those 6 messages every minute on the network? I don't think you can get it working with just M-SEARCH responses, since that is not the way Toon discovers a Hue bridge.

So basiclly, we need your script to broadcast (like the hueUpnp.py script) 6 messages to connect to the Toon...

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b::upnp:rootdevice

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b::upnp:rootdevice

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: uuid:2f402f80-da50-11e1-9b23-00178818572b
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: uuid:2f402f80-da50-11e1-9b23-00178818572b
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.2.102:80/description.xml
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1
NTS: ssdp:alive
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-00178818572b

mariusmotea · 2017-07-26T19:52:06Z

Please check the develop branch, i add a new service for ssdp broadcast. This is experimental, i did not perform any tests.

kurniawan77 · 2017-07-26T20:04:57Z

I will give it a try!
ewjmulder mentioned that there are 3 different messages repeated twice within 60 seconds. I only see one message in the script, is this correct?

mariusmotea · 2017-07-26T20:13:45Z

We can do improvements if is still not working. Be aware that Toon may not listen permanently for ssdp broadcast messages, maybe it need a few minutes to detect the bridge emulator.

kurniawan77 · 2017-07-26T21:01:43Z

Script is running fine. I can see the broadcasts every 10 seconds. But still nothing on the Toon's side... as expected. Waited a few minutes, rebooted then waited a few minutes again.

I believe it needs the other two messages in the same order as i posted this afternoon.

mariusmotea · 2017-07-27T08:02:13Z

Check again the develop branch, i made a new commit. Now theoretically it broadcast this:

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.10.200:80/description.xml
SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
NT: uuid:2f402f80-da50-11e1-9b23-b827ebc76b26
USN: uuid:2f402f80-da50-11e1-9b23-b827ebc76b26

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.10.200:80/description.xml
SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-b827ebc76b26

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://192.168.10.200:80/description.xml
SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-b827ebc76b26::upnp:rootdevice

^Cserver stopped
config saved
pi@raspberrypi:~ $

kurniawan77 · 2017-07-27T12:40:06Z

Nice... I haven't try it yet. Will do this evening. Can't wait to get back home...

I'm not sure about this but isn't it better to simulate the messages the exact same way as those 6 messages above? I will report back as soon as i find out.

mariusmotea · 2017-07-27T13:27:11Z

very likely Toon is not so strict regarding ssdp messages and just one of that 3 formats must be the one that Toon is looking in order to trigger Hue Discovery.

kurniawan77 · 2017-07-27T17:43:01Z

Well just tried it... still not connecting. So i guess it needs all six in the exact order like ewjmulder presented.

mariusmotea · 2017-07-27T20:43:30Z

Ok, i update again to have the same order. Problem is that we cannot be sure the correct delay is one message on every 10 seconds. Maybe these must be sent two by two with 20 seconds delay.

mariusmotea · 2017-07-28T09:34:58Z

I execute a ssdp sniffer and i use split diff to check the differences from ewjmulder output with hue emulator and i found one that can be a problem:

SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.16.0
instead of:
SERVER: FreeRTOS/6.0.5, UPnP/1.0, IpBridge/0.1

kurniawan77 · 2017-07-28T09:39:53Z

That crossed my mind this morning too...

And i see ewjmulder uses a Hue mac address in his script and description.xml too...

mariusmotea · 2017-07-28T10:30:36Z

Setup same mac address as ewjmulder on rpi interface and you will have identical ssdp notify requests. The only thing that we still not know is the delay between request, 6 request every minute has lot of possible options.

kurniawan77 · 2017-07-28T12:15:34Z

Will change the mac address again and report back.
About the delay i have asked ewjmulder... waiting for reply. I also had a look in his UpnpSender.java
and from what i can see there is that he uses 10 ms between each message. correct me if i'm wrong... and he uses a sleep after the batch of messages but i can't seem to find for how long.

kurniawan77 · 2017-07-30T19:39:49Z

hi there,

I have been trying this weekend. I can't get any response from the Toon. I have tried different delay times and changed my Mac address to same as ewjmulder... nothing!

Some how i messed up somewhere and couldn't even connect with the official hue app. Hue app said that the hue bridge needed an update and there was no way i could control the lights. So i started all over, i had a backup of the config files while it was working before and put those back. After this i got back the connection with hue app but... it still says that i need to update the hue bridge...

just reporting

mariusmotea · 2017-07-30T20:14:08Z

Hi,

If you update Hue app then is possible to force you to update also the bridge to latest version, it happened once #11 , but now i don't see any recent update on my phone. Please check if there are differences in your config.json => config key and the one from the repo. I suspect differences can be at api or swversion keys. If is still not working check if will ask for update with default config.json.
Regarding Toon, try to broadcast ssdp messages one every second (hope this will not flood some network devices). If still not working then we must find help from somebody that own an original Hue Bridge and can sniff the ssdp on his network. I consider to buy one but in about one, two months.

kurniawan77 · 2017-08-02T12:44:24Z

Yes it was indeed because of differences in config.json... somehow it was one i downloaded from develop branche...

kurniawan77 · 2017-08-08T12:38:44Z

Alright... for the sake of this project, i bought myself a Hue bridge v2 secondhand and started to sniff the network once it was installed.

I can confirm that, as we know now, that there are 3 different ssdp broadcast messages sent. They look like this from the real bridge:

Message0:

    NOTIFY * HTTP/1.1\r\n
    HOST: 239.255.255.250:1900\r\n
    CACHE-CONTROL: max-age=100\r\n
    LOCATION: http://192.168.129.238:80/description.xml\r\n
    SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.20.0\r\n
    NTS: ssdp:alive\r\n
    hue-bridgeid: 001788FFFE2FC864\r\n
    NT: upnp:rootdevice\r\n
    USN: uuid:2f402f80-da50-11e1-9b23-0017882fc864::upnp:rootdevice\r\n
    \r\n

Message1:

    NOTIFY * HTTP/1.1\r\n
    HOST: 239.255.255.250:1900\r\n
    CACHE-CONTROL: max-age=100\r\n
    LOCATION: http://192.168.129.238:80/description.xml\r\n
    SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.20.0\r\n
    NTS: ssdp:alive\r\n
    hue-bridgeid: 001788FFFE2FC864\r\n
    NT: uuid:2f402f80-da50-11e1-9b23-0017882fc864\r\n
    USN: uuid:2f402f80-da50-11e1-9b23-0017882fc864\r\n
    \r\n

Message2:

    NOTIFY * HTTP/1.1\r\n
    HOST: 239.255.255.250:1900\r\n
    CACHE-CONTROL: max-age=100\r\n
    LOCATION: http://192.168.129.238:80/description.xml\r\n
    SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.20.0\r\n
    NTS: ssdp:alive\r\n
    hue-bridgeid: 001788FFFE2FC864\r\n
    NT: urn:schemas-upnp-org:device:basic:1\r\n
    USN: uuid:2f402f80-da50-11e1-9b23-0017882fc864\r\n
    \r\n

I can see that we are missing one line: hue-bridgeid: 001788FFFE2FC864\r\n and maybe the last \r\n.

TIMING

As for the timing... with wireshark i can see that each message is indeed broadcasted twice and all six messages rapidly after each other within 1 second. See image below

This also goes for the ssdp search messages. Here we also have three different search messages:

Search 0:

HTTP/1.1 200 OK\r\n
    HOST: 239.255.255.250:1900\r\n
    EXT:\r\n
    CACHE-CONTROL: max-age=100\r\n
    LOCATION: http://192.168.129.238:80/description.xml\r\n
    SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.20.0\r\n
    hue-bridgeid: 001788FFFE2FC864\r\n
    ST: upnp:rootdevice\r\n
    USN: uuid:2f402f80-da50-11e1-9b23-0017882fc864::upnp:rootdevice\r\n
    \r\n

Search 1:

HTTP/1.1 200 OK\r\n
   HOST: 239.255.255.250:1900\r\n
   EXT:\r\n
   CACHE-CONTROL: max-age=100\r\n
   LOCATION: http://192.168.129.238:80/description.xml\r\n
   SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.20.0\r\n
   hue-bridgeid: 001788FFFE2FC864\r\n
   ST: uuid:2f402f80-da50-11e1-9b23-0017882fc864\r\n
   USN: uuid:2f402f80-da50-11e1-9b23-0017882fc864\r\n
   \r\n

Search 2:

HTTP/1.1 200 OK\r\n
    HOST: 239.255.255.250:1900\r\n
    EXT:\r\n
    CACHE-CONTROL: max-age=100\r\n
    LOCATION: http://192.168.129.238:80/description.xml\r\n
    SERVER: Linux/3.14.0 UPnP/1.0 IpBridge/1.20.0\r\n
    hue-bridgeid: 001788FFFE2FC864\r\n
    ST: urn:schemas-upnp-org:device:basic:1\r\n
    USN: uuid:2f402f80-da50-11e1-9b23-0017882fc864\r\n
    \r\n

These three ssdp searches are sent rapidly after each other within 1 second for 8 times (8 seconds).

In visual:

and here's an excel export from this afternoon
export.xlsx

I will also be trying HA-Bridge https://github.com/bwssytems/ha-bridge which also simulates Hue bridge. From what i can read is that some say HA-Bridge connects with Toon, but some don't.....

mariusmotea · 2017-08-08T13:02:37Z

Cool, in a few hours i will be home and i will perform a new commit

mariusmotea · 2017-08-08T16:38:42Z

I check your traffic sniffer and i come to following conclusion:
What you have there marked with "search [0-2]" are replies to M-SEARCH queries from 192.168.129.236 (windows, chrome, etc perform these automatically on network). Toon don't perform any M-SEARCH query so this is not important for us. Looking at "messages [0-2]" i found they are very close to what we already done, except what you already discover. I update the developer branch to have identical broadcast message.
BTW. if you paired Toon with original bridge you must perform factory reset (i remember ewjmulder said something like this). Don't forget to monitor emulator output, maybe you will find the GET request to /desctiption.xml or maybe an M-SEARCH from Toon.

kurniawan77 · 2017-08-10T06:33:08Z

Tried yesterday evening... no luck...

Also performed a factory reset because Toon kept finding the real hue bridge, even if it was powerless! Somehow it saves it somewhere, didn't even paired with it, just did a search, so i had to reconfigure the Toon, took some time. After that i still couldn't connect with HueEmulator. I'm clueless... but i will keep trying.

kurniawan77 · 2017-08-10T06:35:11Z

o and i didn't even get any response from Toon on the emulator output. I did from other devices. I didn't snif the Toon yesterday, maybe this evening.

mariusmotea · 2017-08-10T07:39:55Z

this is strange because ssdp broadcast messages must be identical now. I suggest you to change the mac address on RPI because maybe Toon is filtering for ssdp messages that are coming just from some know devices. This is done by appending in /boot/cmdline.txt the following text "smsc95xx.macaddr=00-17-88-2f-c8-65". Here i increase with 1 the mac from your original bridge. If there are other differences then wireshark must find them. You can also play by putting the same mac address on RPI as original bridge and after toon detect it just made the switch on network with rpi so the pair will be done with emulator.

kurniawan77 · 2017-08-11T12:32:08Z

still no luck... changed mac address as you advised (1 up), will do the exact mac address as the real bridge tonight. And maybe do the swap... but that's not a good solution imho (if it works).

Nice way to change the mac address btw with cmdline.txt. It has to be done with " : " instead of " - ".

o and occasionally i do get an exception, the ip address is from my router...

kurniawan77 · 2017-08-12T13:28:28Z

oke... just reporting some strange behavior from the real Hue bridge i simply don't understand. But i just wanted to show a difference from the HueEmulator.

Here's what's happening:

I was googling if there were problems connecting to a real hue bridge and found out about an url people could use to see if the hue is on the same network. This url looks like this:

https://www.meethue.com/api/nupnp (You have to use the whole url including https://)

So i gave it a try and while doing this, the real bridge was unplugged from power, Toon was on and HueEmulator was running. I got a reply in my browser (chrome) which looked like:

[]

Ofcourse, because there is no real hue bridge. Then i wanted to try this with the real Hue bridge on, so i powered off the Toon (just in case) and powered on the real Hue bridge, went back to my browser and refreshed the url. It replied with:

[{"id":"001788fffe2fc864","internalipaddress":"192.168.129.238"}] (How funny)

But now comes the part i don't understand!

After i got the reply i unplugged the real Hue bridge from power. Waited a few minutes, powered on the Toon, went back to my laptop and refreshed the url on the browser, guess what. I got the same reply back...: [{"id":"001788fffe2fc864","internalipaddress":"192.168.129.238"}] Isn't that strange!? The real hue bridge is powered off! So maybe it's the cache in chrome browser, opened up Edge and did the url, same result:

So, maybe this is saved local?
(Now i'm getting the feeling this is the same behaviour the Toon gets when it finds the real Hue and keeps finding it even if the bridge is powered off.)
I grabbed my phone and tablet, entered the url and get the same reply!!!
After a hour and after writing this whole story i tried once again on all devices at home it finally disappeared!

So i am wondering how this works even if the bridge is off, but nevermind about it. More important is that the real bridge has this behavior and the HueEmulator doesn't. I have a feeling that the Toon somehow needs this reply... but i could be wrong. This is how the Toon looks like with wireshark:

mariusmotea · 2017-08-12T14:04:40Z

Here we can have a problem that can be very hard to fix. I start to read about that url and it seems this is a service provided by Philips on their servers. Is know that there is available also a second api (the remote api) used by hue bridge to connect to Philips servers and this is not public. Toon is possible to use this service because is very easy to implement (just a curl to that url will return the private ip and mac address of the local bridge). Chrome don't cache anything here, information is cached on remote servers and cleared after connection to real bridge is lost with in few minutes. When there is a request to that url that come from same public ip like the one that bridge was using to register to online services then the output will be that json.
What we can do in this case:
redirect via static dns record on your router https://www.meethue.com to a local webserver that will output the valid json for emulator bridge on GET request to /api/nupnp path. Out big problem is that the url is https and not http. If Toon will verify the entire ssl certificate chain, then it will see this is unsecured and stop there. I believe this is the last chance if Philips will not made the remote api public.

mariusmotea · 2017-08-12T14:31:56Z

Easy way to test if Toon use this method and don't perform ssl check:
https://gist.github.com/dergachev/7028596

generate the self signed certificate with the command provided and replace
'localhost', 4443
with
'0.0.0.0', 443

if you will see any requests from Toon after adding the static dns record then this can be hacked, if not then maybe Toon don't use this method to discover the bridge or Toon perform ssl check and it fail.

kurniawan77 · 2017-09-05T13:03:02Z

Hi there,
sorry for my late respond. I was on vacation.
I am putting this on hold for now. Reason for this is that i want to go on further with this beautiful project and build my house with it. It's cheap and works perfect! Also made an ESP Mi-light hub for my (for now) existing lights. I will pick this up when the rest is implemented.

mariusmotea · 2017-09-05T13:28:35Z

No problem. For now i prepare the integration with Raspbee module for raspberry pi that will control the zigbee lights directly, and will work will work with zigbee switches and sensors.

mariusmotea · 2017-10-06T09:14:09Z

Hi @kurniawan77

Can you test ESP826Emulator with Toon? One user is saying that all Toon devices must work with it
Issue is this: probonopd/ESP8266HueEmulator#76

Marius.

kurniawan77 · 2017-10-06T12:16:40Z

Hi there @mariusmotea

Sure, will do and report back.

kurniawan77 · 2017-10-06T19:00:41Z

Bad news... not working

ghost · 2018-08-31T15:23:05Z

Ok, I'm going to close this as it has clearly been put on hold by everyone. Also it looks like it is the remote API that is preventing this from moving forward. So until we have a solution for the remote API it looks like Toon will not work. Feel free to reopen this @kurniawan77 if you wan't to work on it again.

Fix Ikea Tradfri lights not updating

Update from mainstream

kurniawan77 changed the title ~~No connection with E~~ No connection with Eneco Toon (Quby) Jul 17, 2017

kurniawan77 mentioned this issue Jul 25, 2017

Help us ewjmulder/hue-bridge-simulator#2

Closed

This was referenced Oct 2, 2017

Anyone tried to use ESP8266HueEmulator with Amazon Alexa? probonopd/ESP8266HueEmulator#62

Open

Amazon Echo #1

Closed

HueEmulator not picked up by Quby (Toon eneco dutch) mabey others ? probonopd/ESP8266HueEmulator#76

Open

ghost assigned mariusmotea Aug 29, 2018

ghost added the Add Hub Support label Aug 29, 2018

ghost closed this as completed Aug 31, 2018

xdjoshuaaz pushed a commit to xdjoshuaaz/diyHue that referenced this issue Jul 6, 2019

Merge pull request mariusmotea#23 from cwildfoerster/master

d67bdbb

Fix Ikea Tradfri lights not updating

grywnn pushed a commit to grywnn/diyHue that referenced this issue Nov 15, 2019

Merge pull request mariusmotea#23 from diyhue/master

4fefb10

Update from mainstream

This issue was closed.

No connection with Eneco Toon (Quby) #23

No connection with Eneco Toon (Quby) #23

Comments

kurniawan77 commented Jul 17, 2017 • edited Loading

kurniawan77 commented Jul 17, 2017 • edited Loading

mariusmotea commented Jul 17, 2017

mariusmotea commented Jul 18, 2017

kurniawan77 commented Jul 19, 2017

mariusmotea commented Jul 19, 2017 • edited Loading

kurniawan77 commented Jul 23, 2017 • edited Loading

mariusmotea commented Jul 23, 2017 • edited Loading

mariusmotea commented Jul 25, 2017

kurniawan77 commented Jul 25, 2017

mariusmotea commented Jul 25, 2017

kurniawan77 commented Jul 25, 2017

kurniawan77 commented Jul 26, 2017

mariusmotea commented Jul 26, 2017

kurniawan77 commented Jul 26, 2017

mariusmotea commented Jul 26, 2017

kurniawan77 commented Jul 26, 2017

mariusmotea commented Jul 27, 2017

kurniawan77 commented Jul 27, 2017

mariusmotea commented Jul 27, 2017

kurniawan77 commented Jul 27, 2017

mariusmotea commented Jul 27, 2017

mariusmotea commented Jul 28, 2017 • edited Loading

kurniawan77 commented Jul 28, 2017

mariusmotea commented Jul 28, 2017

kurniawan77 commented Jul 28, 2017

kurniawan77 commented Jul 30, 2017

mariusmotea commented Jul 30, 2017

kurniawan77 commented Aug 2, 2017

kurniawan77 commented Aug 8, 2017 • edited Loading

mariusmotea commented Aug 8, 2017

mariusmotea commented Aug 8, 2017

kurniawan77 commented Aug 10, 2017

kurniawan77 commented Aug 10, 2017

mariusmotea commented Aug 10, 2017

kurniawan77 commented Aug 11, 2017

kurniawan77 commented Aug 12, 2017 • edited Loading

mariusmotea commented Aug 12, 2017

mariusmotea commented Aug 12, 2017

kurniawan77 commented Sep 5, 2017

mariusmotea commented Sep 5, 2017

mariusmotea commented Oct 6, 2017

kurniawan77 commented Oct 6, 2017

kurniawan77 commented Oct 6, 2017

ghost commented Aug 31, 2018

kurniawan77 commented Jul 17, 2017 •

edited

Loading

kurniawan77 commented Jul 17, 2017 •

edited

Loading

mariusmotea commented Jul 19, 2017 •

edited

Loading

kurniawan77 commented Jul 23, 2017 •

edited

Loading

mariusmotea commented Jul 23, 2017 •

edited

Loading

mariusmotea commented Jul 28, 2017 •

edited

Loading

kurniawan77 commented Aug 8, 2017 •

edited

Loading

kurniawan77 commented Aug 12, 2017 •

edited

Loading