UPnP SSDP discovery not working using UDP multicast #3

truedat101 · 2017-09-25T17:33:12Z

I've implemented this same logic in several other languages, but couldn't get it working in nim. I'm assuming it is my lack of understanding of the libraries or the low-level networking.

Method in question:
https://github.com/IoTone/huenim/blob/master/src/huenim.nim#L136
findHueHubsViaUPnP()

To reproduce:

Need a HUE hub with lights configured
setup a username/apikey

Call findHueHubsViaUPnP()
It should return some response for any other devices on the network that match the criteria. However, no devices respond, not a single response. I can confirm that the request goes out (using network tools to monitor UDP multicast messages).

truedat101 · 2017-09-25T17:37:07Z

Assumption is I have some problems in the use of the nim apis. If it helps I can post some C code that does the same thing.

dom96 · 2017-09-25T19:52:42Z

Yeah, please post the C code and also the output that this code gives.

truedat101 · 2017-09-25T20:01:13Z

Will dig up the code and post that output.

truedat101 · 2017-09-26T05:58:26Z

I have a bad example I put together in C here:
https://pastebin.com/CvmqvV2j

The bad example doesn't use the query to do a sendto first before listening for a response. I noticed that regardless of my query, I will get a response from my network which I can easily filter out the key tell-tale signs this is a Hue Hub.

Here are some responses from this code below. The giveaway that this is a HUE hub is the presence of a LOCATION value that ends with discovery.xml, for example:
LOCATION: http://10.0.0.222:80/description.xml

One of the approaches people take to find their hub(s) is to enumerate all ip addresses in the range of the local network, and try to hit port 80/description.xml. However that's sort of a waste. If the UPnP approach works, it should yield a narrow set of devices that will respond, and from those you filter based on the LOCATION. It's ugly either way, but the multicast approach should be more efficient.

Opening datagram socket....OK.
Setting SO_REUSEADDR...OK.
Binding datagram socket...OK.
Adding multicast group...OK.
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5::upnp:rootdevice

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5::upnp:rootdevice

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: uuid:2f402f80-da50-11e1-9b23-0017881a73b5
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: uuid:2f402f80-da50-11e1-9b23-0017881a73b5
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: upnp:rootdevice
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857::upnp:rootdevice
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: urn:schemas-upnp-org:device:InternetGatewayDevice:1
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857::urn:schemas-upnp-org:device:InternetGatewayDevice:1
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

33119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: urn:schemas-upnp-org:device:WANConnectionDevice:1
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d859::urn:schemas-upnp-org:device:WANConnectionDevice:1
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337



"

truedat101 · 2017-09-26T05:59:10Z

In the Java version of this, it worked like a charm. The nim version I would never get a response.

dom96 · 2017-09-26T15:35:14Z

Hrm, all I can is guess, but this part is missing from your Nim code (isn't it?):

if (setsockopt(sd, IPPROTO_IP, IP_ADD_MEMBERSHIP, (char *)&group, sizeof(group)) < 0)

truedat101 · 2017-09-26T16:09:31Z

Thanks. I'll revisit this and try that, spent some time on different variations of the discover initially, and eventually switched to the nupnp implementation. UPnP has a bad rap because of security concerns, but this is still useful for a number of IoT type devices that are out there.

enthus1ast · 2018-02-16T16:09:25Z

@truedat101 if this is still relevant you could have a look at https://github.com/enthus1ast/nimMulticast

truedat101 · 2018-02-16T16:13:20Z

@enthus1ast actually yeah, awesome. I spent a fair amount of time chasing down what I was doing wrong , so will have a look at your lib.

truedat101 · 2018-10-20T15:16:06Z

Will have a look at this module nimMulticast.

truedat101 · 2018-10-20T15:16:20Z

Finally have some time to look at this.

enthus1ast · 2018-10-20T15:53:28Z

just skimmed through your code:
bindAddr(socket, Port(1900), "239.255.255.250")
afaik for multicast there is no need to bind to the multicast address.
The multicast enabled socket will receive every datagram send to the multicast group.

truedat101 · 2018-10-21T05:03:36Z

Good point. Need to get back into this.

truedat101 · 2018-10-23T07:13:14Z

Hmm, tried socket.bindAddr(Port(1900)) , but still just get back empty response. I have to remember how I was snooping the network for UPnP discovery. I've been through the code several times and didn't find a solution.

Need to try the suggestion #3 (comment) from Dominik.

enthus1ast · 2018-10-23T10:34:31Z

this principally is what multicast joinGroup does.
So to receive multicast datagrams you have to:

create a socket
bindAddr (bind to your interface address not the multicast group!)
socket.joinGroup("239.255.255.250")
when joinGroup returns true, the socket should be able to receive multicast.
you can test this by sending to the multicast group (the sender socket does not need to be in the multicast group) so sending a datagram with ncat should be enough:

[david@eb ~]$ ncat -4 --udp 239.255.255.250 1900
this should be visible in your testprogram

enthus1ast · 2018-10-23T10:48:11Z

Something like this should work:

bindAddr(socket, Port(1900))
let group = "239.255.255.250"
assert true == socket.joinGroup(group)
let res = socket.sendTo(group, Port(1900),cstring(upnpquery),upnpquery.len.cint)
# [...]
let bytesReceived = recvFrom(socket, data, recvlen, recvaddr, recvport)

truedat101 · 2018-10-23T19:13:04Z

Great, thank you for the tips. I'll jump onto this tonight. As mentioned, I have this all working reliably in my java code and also C code version, but really wanted to make this a no-brainer thing to work with UPnP w/ NIM, and eliminate the need to do discovery using the Hue's funky nupnp solution that requires a proper internet connection.

truedat101 · 2018-10-24T05:58:40Z

So I've added 0.1.2 version of multicast to the project. Strange error when I compile.

multicast-0.1.2/multicast.nim(54, 6) Error: undeclared identifier: 'IPv4'

This should be present in IPAddressFamily. Anyway, is this something obvious I've missed, or should I file a ticket on multicast.

enthus1ast · 2018-10-24T10:44:50Z

@truedat101 strange that this works for me, since this is a pure enum mhh.
Nonetheless i pushed a new version, please try if it now compiles for you.

truedat101 · 2018-10-24T14:20:55Z

@enthus1ast Thanks I'll give this a look. I guess the only thing I can think of that might be different is I'm on Mac OS X for this particular project and using num 0.17.0.

enthus1ast · 2018-10-24T14:25:12Z

i bet this issue is because of 0.17 but i see another issue comeing: it might be that IP_ADD_MEMBERSHIP and IP_DROP_MEMBERSHIP int values differs from posix (they differ from old windows to new windows and linux.
I might have to add these values for mac. Unfortunately i have no access to a mac. So please report if its working :)

truedat101 · 2018-10-24T14:38:32Z

Ahh,ok, that does make sense. I can continue testing on the mac. At least that can get resolved through testing. I can move to a later release as well.

truedat101 · 2018-10-25T04:57:44Z

Great upgraded to multicast module 0.1.3. Build is fine.

On the run, I get:

Traceback (most recent call last)
runTests.nim(12)         runTests
huenim.nim(155)          findHueHubsViaUPnP
system.nim(3613)         failedAssertImpl
system.nim(3605)         raiseAssert
system.nim(2724)         sysFatal

    Unhandled exception: true == joinGroup(socket, group, 255)
assert true == socket.joinGroup(group)

enthus1ast · 2018-10-27T09:35:24Z

@truedat101 i've pushed a new branch of multicast: https://github.com/enthus1ast/nimMulticast/tree/freebsdAndMacos

i still have no access to macos but i've installed a freebsd and tested the branch on it.
I've also fixed the example (edit: i know the example was working with an older nim version, so it might be, if you still use 0.17 that i've just broke the example for you) so to test if multicast works on macos you could just run the tests/example.nim on two different machines.

Please report if macos works now, then i'll merge it into master :)

truedat101 · 2018-10-28T14:22:22Z

Apologies for the delay. I am sad to report my mac died. Awaiting its return from the shop and then I'll jump back onto testing that.

truedat101 · 2018-11-17T06:23:50Z

@enthus1ast mac is back. Going to give this a go.

truedat101 mentioned this issue Sep 25, 2017

Add package huenim nim-lang/packages#588

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UPnP SSDP discovery not working using UDP multicast #3

UPnP SSDP discovery not working using UDP multicast #3

truedat101 commented Sep 25, 2017 •

edited

Loading

truedat101 commented Sep 25, 2017

dom96 commented Sep 25, 2017

truedat101 commented Sep 25, 2017

truedat101 commented Sep 26, 2017

truedat101 commented Sep 26, 2017

dom96 commented Sep 26, 2017

truedat101 commented Sep 26, 2017

enthus1ast commented Feb 16, 2018

truedat101 commented Feb 16, 2018

truedat101 commented Oct 20, 2018

truedat101 commented Oct 20, 2018

enthus1ast commented Oct 20, 2018

truedat101 commented Oct 21, 2018

truedat101 commented Oct 23, 2018

enthus1ast commented Oct 23, 2018 •

edited

Loading

enthus1ast commented Oct 23, 2018 •

edited

Loading

truedat101 commented Oct 23, 2018

truedat101 commented Oct 24, 2018

enthus1ast commented Oct 24, 2018

truedat101 commented Oct 24, 2018 •

edited

Loading

enthus1ast commented Oct 24, 2018

truedat101 commented Oct 24, 2018

truedat101 commented Oct 25, 2018 •

edited

Loading

enthus1ast commented Oct 27, 2018 •

edited

Loading

truedat101 commented Oct 28, 2018

truedat101 commented Nov 17, 2018

UPnP SSDP discovery not working using UDP multicast #3

UPnP SSDP discovery not working using UDP multicast #3

Comments

truedat101 commented Sep 25, 2017 • edited Loading

truedat101 commented Sep 25, 2017

dom96 commented Sep 25, 2017

truedat101 commented Sep 25, 2017

truedat101 commented Sep 26, 2017

truedat101 commented Sep 26, 2017

dom96 commented Sep 26, 2017

truedat101 commented Sep 26, 2017

enthus1ast commented Feb 16, 2018

truedat101 commented Feb 16, 2018

truedat101 commented Oct 20, 2018

truedat101 commented Oct 20, 2018

enthus1ast commented Oct 20, 2018

truedat101 commented Oct 21, 2018

truedat101 commented Oct 23, 2018

enthus1ast commented Oct 23, 2018 • edited Loading

enthus1ast commented Oct 23, 2018 • edited Loading

truedat101 commented Oct 23, 2018

truedat101 commented Oct 24, 2018

enthus1ast commented Oct 24, 2018

truedat101 commented Oct 24, 2018 • edited Loading

enthus1ast commented Oct 24, 2018

truedat101 commented Oct 24, 2018

truedat101 commented Oct 25, 2018 • edited Loading

enthus1ast commented Oct 27, 2018 • edited Loading

truedat101 commented Oct 28, 2018

truedat101 commented Nov 17, 2018

truedat101 commented Sep 25, 2017 •

edited

Loading

enthus1ast commented Oct 23, 2018 •

edited

Loading

enthus1ast commented Oct 23, 2018 •

edited

Loading

truedat101 commented Oct 24, 2018 •

edited

Loading

truedat101 commented Oct 25, 2018 •

edited

Loading

enthus1ast commented Oct 27, 2018 •

edited

Loading