Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mDNS Support #1376

Open
opichals opened this issue Apr 18, 2018 · 18 comments
Open

mDNS Support #1376

opichals opened this issue Apr 18, 2018 · 18 comments

Comments

@opichals
Copy link
Contributor

opichals commented Apr 18, 2018

It would be nice to develop at least a minimal mDNS support.

Features:

  • respond to hostname.local mDNS queries
  • respond to _services._dns-sd._udp.local queries
  • should provide callback-based interface for providing specific service query responders
@opichals
Copy link
Contributor Author

opichals commented Apr 18, 2018

At the moment there is this hacked-up implementation gist.

@wilberforce
Copy link
Member

wilberforce commented May 17, 2018

@opichals

At this point I would like to get the ESP32 to do this -> respond to hostname.local mDNS queries
rather than use the ESP-IDF libs would like to do in js. I tried your gist - wow - lots of output!

Any reason this is closed?

@opichals
Copy link
Contributor Author

opichals commented May 17, 2018

@wilberforce This issue is not closed, only the ESP8266 specific issue is as the intention is to have a .js module for it which is not board specific.

The gist was partially used to debug the dgram implementation so that's why it's got so much output. I have been working on a callback-based version of that recently. I am going to update the gist as soon as I get to it.

@opichals
Copy link
Contributor Author

opichals commented May 17, 2018

@wilberforce Gist updated with a version that doesn't hold the whole parsed DNS packet in memory and rather issues callback calls for every record.

As I mentioned I have been hacking on this recently and I still intend to extend it to provide the features from this issue description.

@wilberforce
Copy link
Member

wilberforce commented May 17, 2018

Thanks for working on this. I had to hard code the hostname wifi.gethostname is not implemented on the esp32 - I guess you on the esp8266?

I set the hostname to esp would you expect http://esp.local/ to work yet?

Have some debug output - looks like apple TV is giving unexpected output:

"message": "Field or method \"NaN\" does not already exist, and can't create it on Uint8Array",

MSG { "tid": 0, "flags": 0, "questions": 1, "answerRRs": 2,
  "authority": 0, "additionalRRs": 1,
  "records": [  ]
 }
REC {"name":"_sleep-proxy._udp.local","type":12,"clazz":1,"len":29}
REC {"name":"_sleep-proxy._udp.local","type":12,"clazz":1,"len":35,"ttl":4497,"data":[20,55,48,45,51,53,45,54,48,45,54,51,46,49,32,76,111,117,110,103,101,192,12]}
REC {"name":"_sleep-proxy._udp.local","type":12,"clazz":1,"len":40,"ttl":4499,"data":[25,55,48,45,51,53,45,54,48,45,54,51,46,49,32,70,97,109,105,108,121,32,82,111,111,109,192,12]}
REC {"name":"","type":41,"clazz":1440,"len":29,"ttl":4500,"data":[0,4,0,14,0,240,172,188,50,104,157,140,172,188,50,104,157,138]}
MEM {"free":2012,"usage":488,"total":2500,"history":592,"gc":0,"gctime":2.317}
memory {"free":1956,"usage":544,"total":2500,"history":592,"gc":0,"gctime":2.367}
s:message 192.168.15.9 5353
MSG { "tid": 0, "flags": 33792, "questions": 0, "answerRRs": 23,
  "authority": 0, "additionalRRs": 1,
  "records": [  ]
 }
REC {"name":"Family Room._companion-link._tcp.local","type":16,"clazz":32769,"len":95,"ttl":4500,"data":[22,114,112,66,65,61,65,67,58,66,67,58,51,50,58,54,56,58,57,68,58,56,66,9,114,112,70,108,61,48,120,56,50,11,114,112,86,114,61,49,52,48,46,51,49]}
REC {"name":"_services._dns-sd._udp.local","type":12,"clazz":1,"len":37,"ttl":4500,"data":[192,24]}
REC {"name":"_companion-link._tcp.local","type":12,"clazz":1,"len":14,"ttl":4500,"data":[192,12]}
REC {"name":"Family Room._device-info._tcp.local","type":16,"clazz":1,"len":50,"ttl":4500,"data":[12,109,111,100,101,108,61,74,52,50,100,65,80]}
REC {"name":"ACBC32689D8C@Family Room._raop._tcp.local","type":16,"clazz":32769,"len":232,"ttl":4500,"data":[10,99,110,61,48,44,49,44,50,44,51,7,100,97,61,116,114,117,101,8,101,116,61,48,44,51,44,53,21,102,116,61,48,120,53,65,55,70,70,70,70,55,44,48,120,53,53,70,68,69,8,115,102,61,48,120,50,52,52,8,109,100,61,48,44,49,44,50,13,97,109,61,65,112,112,108,101,84,86,53,44,51,67,112,107,61,55,55,52,53,57,51,97,55,52,49,97,56,55,98,98,54,97,57,57,101,50,98,53,51,98,48,49,54,54,54,52,50,49,56,48,98,49,100,102,98,101,102,99,48,99,48,99,57,56,54,55,56,102,101,98,102,98,102,99,50,99,54,50,98,6,116,112,61,85,68,80,8,118,110,61,54,53,53,51,55,9,118,115,61,51,54,53,46,53,51,7,111,118,61,49,49,46,51,4,118,118,61,50]}
REC {"name":"_services._dns-sd._udp.local","type":12,"clazz":1,"len":14,"ttl":4500,"data":[192,233]}
REC {"name":"_raop._tcp.local","type":12,"clazz":1,"len":14,"ttl":4500,"data":[192,208]}
REC {"name":"Family Room._airplay._tcp.local","type":16,"clazz":32769,"len":60,"ttl":4500,"data":[5,97,99,108,61,48,26,100,101,118,105,99,101,105,100,61,65,67,58,66,67,58,51,50,58,54,56]}
Error Error {
  "message": "Field or method \"NaN\" does not already exist, and can't create it on Uint8Array",
  "type": "Error",
  "stack": " at line 13 col 33\n            len = bytes[offset++];\n                                ^\nin function \"parseDnsName\" called from line 1 col 41\nvar dnsName = parseDnsName(bytes, offset);\n                                        ^\nin function \"parseRecord\" called from line 3 col 61\n...bytes, ctx.offset, isAnswer);\n                              ^\nin function \"handleRecords\" called from line 20 col 43\n    handleRecords(ctx, packet.answerRRs, 1);\n                                          ^\nin function \"processMessage\" called from line 9 col 18\n                });\n                 ^\n"
 }

@opichals
Copy link
Contributor Author

opichals commented May 18, 2018

For .js development I mostly run espruino binary on a mac. Once sufficiently tuned I try the boards. And yes, mostly ESP8266.

Hm, I guess I will need to print the byte contents before parsing so that we can debug the NaN you are seeing. And there is definitely more stuff to iron out as I am also seeing some sort of mem-leak or something (the jsvLock > 15 assert) under some circumstances.

I have updated the gist a bit more to parse PTR, SRV and TXT records.

@wilberforce
Copy link
Member

wilberforce commented May 18, 2018

I added the debug on ESP32:

function parseDnsName(bytes, offset) {
  console.log({fn:'parseDnsName',offset:offset,len:bytes.length,bytes:bytes});
    var start = offset;

So it looks like the offset is bigger than actual data...

{ "offset": 814, "len": 516,
"bytes": new Uint8Array([11, 70, 97, 109, 105, 108, 121, 32, 82, 111, 111, 109, 15, 95, 99, 111, 109, 112, 97, 110, 105, 111, 110, 45, 108, 105, 110, 107, 4, 95, 116, 99, 112, 5, 108, 111, 99, 97, 108, 0, 0, 16, 128, 1, 0, 0, 17, 148, 0, 45, 22, 114, 112, 66, 65, 61, 65, 67, 58, 66, 67, 58, 51, 50, 58, 54, 56, 58, 57, 68, 58, 56, 66, 9, 114, 112, 70, 108, 61, 48, 120, 56, 50, 11, 114, 112, 86, 114, 61, 49, 52, 48, 46, 51, 49, 9, 95, 115, 101, 114, 118, 105, 99, 101, 115, 7, 95, 100, 110, 115, 45, 115, 100, 4, 95, 117, 100, 112, 192, 45, 0, 12, 0, 1, 0, 0, 17, 148, 0, 2, 192, 24, 192, 24, 0, 12, 0, 1, 0, 0, 17, 148, 0, 2, 192, 12, 11, 70, 97, 109, 105, 108, 121, 32, 82, 111, 111, 109, 12, 95, 100, 101, 118, 105, 99, 101, 45, 105, 110, 102, 111, 192, 40, 0, 16, 0, 1, 0, 0, 17, 148, 0, 13, 12, 109, 111, 100, 101, 108, 61, 74, 52, 50, 100, 65, 80, 24, 65, 67, 66, 67, 51, 50, 54, 56, 57, 68, 56, 67, 64, 70, 97, 109, 105, 108, 121, 32, 82, 111, 111, 109, 5, 95, 114, 97, 111, 112, 192, 40, 0, 16, 128, 1, 0, 0, 17, 148, 0, 189, 10, 99, 110, 61, 48, 44, 49, 44, 50, 44, 51, 7, 100, 97, 61, 116, 114, 117, 101, 8, 101, 116, 61, 48, 44, 51, 44, 53, 21, 102, 116, 61, 48, 120, 53, 65, 55, 70, 70, 70, 70, 55, 44, 48, 120, 53, 53, 70, 68, 69, 8, 115, 102, 61, 48, 120, 50, 52, 52, 8, 109, 100, 61, 48, 44, 49, 44, 50, 13, 97, 109, 61, 65, 112, 112, 108, 101, 84, 86, 53, 44, 51, 67, 112, 107, 61, 55, 55, 52, 53, 57, 51, 97, 55, 52, 49, 97, 56, 55, 98, 98, 54, 97, 57, 57, 101, 50, 98, 53, 51, 98, 48, 49, 54, 54, 54, 52, 50, 49, 56, 48, 98, 49, 100, 102, 98, 101, 102, 99, 48, 99, 48, 99, 57, 56, 54, 55, 56, 102, 101, 98, 102, 98, 102, 99, 50, 99, 54, 50, 98, 6, 116, 112, 61, 85, 68, 80, 8, 118, 110, 61, 54, 53, 53, 51, 55, 9, 118, 115, 61, 51, 54, 53, 46, 53, 51, 7, 111, 118, 61, 49, 49, 46, 51, 4, 118, 118, 61, 50, 192, 107, 0, 12, 0, 1, 0, 0, 17, 148, 0, 2, 192, 233, 192, 233, 0, 12, 0, 1, 0, 0, 17, 148, 0, 2, 192, 208, 11, 70, 97, 109, 105, 108, 121, 32, 82, 111, 111, 109, 8, 95, 97, 105, 114, 112, 108, 97, 121, 192, 40, 0, 16, 128, 1, 0, 0, 17, 148, 1, 69, 5, 97, 99, 108, 61, 48, 26, 100, 101, 118, 105, 99, 101, 105, 100, 61, 65, 67, 58, 66, 67, 58, 51, 50, 58, 54, 56])
}
Error Error {
"message": "Field or method "NaN" does not already exist, and can't create it on Uint8Array",
"type": "Error",
"stack": " at line 14 col 33\n len = bytes[offset++];\n ^\nin function "parseDnsName" called from line 1 col 41\nvar dnsName = parseDnsName(bytes, offset);\n ^\nin function "parseRecord" called from line 3 col 61\n...bytes, ctx.offset, isAnswer);\n ^\nin function "handleRecords" called from line 19 col 43\n handleRecords(ctx, packet.answerRRs, 1);\n ^\nin function "processMessage" called from line 9 col 18\n });\n ^\n"
}

 at line 14 col 33
 len = bytes[offset++];
 ^in function "parseDnsName" called from line 1 col 41
 var dnsName = parseDnsName(bytes, offset);
 ^in function "parseRecord" called from line 3 col 61
 ..bytes, ctx.offset, isAnswer);
 ^in function "handleRecords" called from line 19 col 43
 handleRecords(ctx, packet.answerRRs, 1);
 ^in function "processMessage" called from line 9 col 18

Also run on linux. and got this:

REC {
  "name": "BRAVIA-4K-2015-930a430dbc269b6151b66d2470cca3a7._googlecast._tcp.local",
  "type": 33, "clazz": 32769, "len": 57, "ttl": 120,
  "data": { "port": 8009,
    "host": "930a430d-bc26-9b61-51b6-6d2470cca3a7.local"
   }
 }
ASSERT(jsvGetLocks(var) < 15) FAILED AT src/jsvar.c:646
  #1[r12,l2] Object {
    #2[r1,l2] Name String [1 blocks] "\xFF"      #3[r1,l2] Object {
        #6[r1,l2] ASSERT(jsvGetLocks(var) < 15) FAILED AT src/jsvar.c:632
EXITING.

@opichals
Copy link
Contributor Author

opichals commented May 18, 2018

@wilberforce Thanks for the debug info. I am able to reproduce as well. The NaN case seems to be caused by the fact that the dgram message is not a complete DNS packet.

Currently the UDP datagrams are simply cut to the max size of the net->chunkSize length. This would require implementing SO_RCVBUF setup so that boards that have enough RAM could receive the whole data.

As for the jsvGetLocks() assert I am suspecting the parseDnsName() just does too much recursion and would need to be rewritten not to.

@opichals
Copy link
Contributor Author

opichals commented May 18, 2018

I tested the createSocket() recvBufferSize option implementation for linux target only for now (sockopt(SO_RCVBUF)/ in master...opichals:createSocket-recvBufferSize-option but I am not completely sure we could afford iterating over all the connections inside every socketIdle() to find the max of all recvBufferSize options used.

Also I think the SO_RCVBUF support is compile-time option for the ESP-IDF and for the ESP32 and I am not sure about whether it is enabled by default. Not to mention other boards and SDKs.

@gfwilliams What do you think?

@wilberforce
Copy link
Member

wilberforce commented May 18, 2018

The NaN case seems to be caused by the fact that the dgram message is not a complete DNS packet.

Can we be brutal here and in the long case just ignore the data?

Also I think the SO_RCVBUF support is compile-time option for the ESP-IDF and for the ESP32 and I am not sure about whether it is enabled by default.

I can take a look and compile in if necessary

@opichals
Copy link
Contributor Author

opichals commented Jun 8, 2018

The NaN case seems to be caused by the fact that the dgram message is not a complete DNS packet.

Can we be brutal here and in the long case just ignore the data?

Yes, that's what I am planning on doing.

@opichals
Copy link
Contributor Author

opichals commented Jun 20, 2018

@wilberforce Updated the gist to avoid recursion in parseDnsName()(getting rid of the ASSERT(jsvGetLocks(var) < 15) issues) and also to check for the packet size inside.

So hopefully the errors seen earlier are gone.

@opichals
Copy link
Contributor Author

opichals commented Jun 21, 2018

The dgram.createSocket({ recvBufferSize: 1024 }) option landed in master in #1472

@wilberforce
Copy link
Member

wilberforce commented Jun 21, 2018

Currently the UDP datagrams are simply cut to the max size of the net->chunkSize length. This would require implementing SO_RCVBUF setup so that boards that have enough RAM could receive the whole data.

I found the option so can compile into the libs for the ESP-idf in the build tools.

I have had some side issues with sockets - the default mtu is something like 1432 - and by default the buffers are 4x this for each connection - so with 10 connections you are looking at a lot of heap use. What seems to occur is that we run out of heap and sockets so op responding. Same code,appears ok on Linux.

Should I try a build with Mtu 536 like the chunk size and 2x the buffers? Also the number of connections seems to be 10 but the number of sockets 16 - I would have thought these should be the same?

@wilberforce
Copy link
Member

wilberforce commented Jun 26, 2018

#1473

This adds the SO_RSVBUF support in the provisioning script for Esp32

@wilberforce
Copy link
Member

wilberforce commented Dec 16, 2018

@opichals fyi

I have added the native mDns to the esp32 port.

#1585

@opichals
Copy link
Contributor Author

opichals commented Dec 16, 2018

@wilberforce Actually I am nearing the submission of the gist. I have been testing it for a while here for NTP server discovery (the resolvePTR function).

@mikabytes
Copy link

mikabytes commented Dec 17, 2021

It's been three years. Is there any progress on getting mDNS to work for Espruino esp8266?

Only mention in the docs is the use of wifi.setHostname, which was not enough. I can't make outgoing requests to .local domains, nor can the esp8266 be discoverable by avahi-browse -a.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants