discovery: swap mdns lib for grandcat/zeroconf #285

rargulati · 2018-02-27T05:57:09Z

Specifically addressing the request here: #257

This commit makes the minimal amount of changes to switch us from
whyrusleeping/mdns to grandcat/zeroconf. Of note, rather than asking the
host.Host which addrs we can listen on, we push this to the
zeroconf library (which runs across available ifaces).

I haven't had a chance to test this on something complicated to see if it retains the desired behavior (to be honest, still new to this codebase, unsure of what all of those would be!), but tests pass locally.

TODO:

Revert PORT behavior
ensure that grandcat/zerconf dep is in gx; tests pass

rargulati · 2018-02-27T05:58:02Z

p2p/discovery/mdns.go

+	info := strings.Join(e.Text, "|")
+
+	// addripv4 potential issue
+	log.Debugf("Handling MDNS entry: %s:%d %s", e.AddrIPv4[0], e.Port, info)


This is the only decision I wasn't certain of.

Kubuxu · 2018-02-27T06:51:22Z

p2p/discovery/mdns.go

 }

 func (m *mdnsService) pollForEntries(ctx context.Context) {

 	ticker := time.NewTicker(m.interval)
 	for {
+		resolver, err := mdns.NewResolver(nil)


I think it might be beast to put resolver in mdnsService struct and reuse it, instead of creating a new one every iteration.

Yikes, good catch. Fixed.

Kubuxu · 2018-02-27T06:52:58Z

p2p/discovery/mdns.go

 func NewMdnsService(ctx context.Context, peerhost host.Host, interval time.Duration, serviceTag string) (Service, error) {

 	// TODO: dont let mdns use logging...
 	golog.SetOutput(ioutil.Discard)

-	var ipaddrs []net.IP
 	port := 4001


This is old but it is very wrong.

Kubuxu · 2018-02-27T10:21:43Z

p2p/discovery/mdns.go

@@ -165,7 +138,7 @@ func (m *mdnsService) handleEntry(e *mdns.ServiceEntry) {
 	}

 	maddr, err := manet.FromNetAddr(&net.TCPAddr{
-		IP:   e.AddrV4,
+		IP:   e.AddrIPv4[0],


As there is possibility of multiple addresses, I would iterate over them, crate maddr for each and add it to the peerstore.

rargulati · 2018-02-27T18:32:11Z

@Stebalien @Kubuxu I've addressed the feedback in c92f1f6

In regards to the port, what would you recommend here? Skimmed through https://tools.ietf.org/html/rfc6762 and see that 5353 is used for the listener, but not seeing anything for the port that's announcing.

I notice that I'm failing tests due to a gx issue. Any recommendations? I'll need to learn that right quick :)

Kubuxu · 2018-02-27T19:52:15Z

@rargulati It was my mistake, I thought it was the port we were announcing to the world but it is mDNS discovery port.

@Stebalien correct if I am wrong, but currently mDNS only publishes peerID in local network, not the full multiaddr, is that right? If it is so, is there a reason not to publish maddr so we can connect to it? Publishing just peerID sees little fruitless as it won't work without DHT capabilities (in case that bootstrap didn't work).

Stebalien · 2018-03-01T22:26:21Z

Port is the port of the service. Also, we do announce the IP addresses and we convert port + IP into /ip4/IP/tcp/PORT. However, that's... suboptimal. It doesn't account for any other potential transports.

Note: For now, I'd be fine preserving the existing behavior. The primary concern here is upgrading to a working mDNS library (one that doesn't have memory safety issues).

However, we also have a bunch of bugs/inconsistencies that we need to fix. I've written them up here:libp2p/libp2p#28

rargulati · 2018-03-05T00:39:35Z

Just to clarify on final things:

Revert PORT to 4001 - the fix to this and other issues will be addressed as detailed in Future of mDNS discovery libp2p#28
~~- [ ] Get grandcat/zerconf into gx (I'm struggling with this - how do I get a new hash for a new dependency - either of you mind doing a quick how-to in irc/email/here?)~~ @Stebalien handling

Does that sound about right @Stebalien @Kubuxu - I'll get this across the line asap.

Stebalien · 2018-03-05T05:46:28Z

Revert PORT to 4001 - the fix to this and other issues will be addressed as detailed in libp2p/libp2p#28

Actually, restore the previous "figure out what port we're using" code. It's not perfect, but it's better than just assuming port 4001.

Get grandcat/zerconf into gx (I'm struggling with this - how do I get a new hash for a new dependency - either of you mind doing a quick how-to in irc/email/here?)

I'll import this into gx tomorrow. We need to write some proper tutorials but it's on our never ending backlog.

Stebalien · 2018-03-06T06:05:01Z

I'll import this into gx tomorrow. We need to write some proper tutorials but it's on our never ending backlog.

This will actually require updating some of our dependencies so it may take a week (blocked on a bunch of other stuff).

deckbsd · 2018-03-24T17:49:00Z

Hi, do you know when this commit will be done ?

Stebalien · 2018-03-24T18:03:27Z

It been blocked on:

Cutting a new release (0.4.14)
Fixing a namesys issue: Add Opentracing plugin support ipfs/kubo#4506
Updating the datastores and the DHT (in go-ipfs).

Once those are done, I'll finally be able to bubble a few low-level updates through the entire tree. The problem here is that this zeroconf library requires a newer version of golang.org/x/net than we use in go-ipfs.

This commit makes the minimal amount of changes to switch us from whyrusleeping/mdns to grandcat/zeroconf. Of note, rather than asking the host.Host which addrs we're available to listen on, we push this to the zeroconf library (which runs across available ifaces).

1. If we receive multiple ipv4 addresses, ensure that we convert them all to multiaddrs and store them in the peerstore. 2. Ensure we don't allocate a new resolver on every tick. Instead reuse this by storing it in our mdns type. 3. Attempt choosing a port.

Ideally, we'd dynamically announce ports based on the current interfaces. However, we'd like to merge this ASAP.

Stebalien · 2018-07-23T19:42:30Z

Still need to fix #376 but I'd like to punt on that.

Stebalien · 2018-07-23T19:50:57Z

Waiting on switch to the built-in context package cenkalti/backoff#59

Stebalien · 2018-07-24T00:11:48Z

p2p/discovery/mdns.go

-	var out []*net.TCPAddr
-	for _, addr := range ph.Addrs() {
+// Tries to pick the best port.
+func getBestPort(addrs []ma.Multiaddr) (int, error) {


This should be fixed by #376

bigs

looks good, just have some confusion regarding some of the handleEntry function. if that's as intended, feel free to merge.

bigs · 2018-07-24T12:04:29Z

p2p/discovery/mdns.go

-	log.Debugf("Handling MDNS entry: %s:%d %s", e.AddrV4, e.Port, e.Info)
-	mpeer, err := peer.IDB58Decode(e.Info)
+	// pull out the txt
+	info := strings.Join(e.Text, "|")


is this designed to fail if there are less than or more than one entry?

to be clear, because of the decoding afterwards

Hm. Yeah, that looks wrong. We should just take the first entry and ignore the rest.

Stebalien · 2018-07-25T04:32:34Z

Blocked on grandcat/zeroconf#38. Fixing that was the reason we went down this road in the first place so there's no real reason to merge this till we get that fixed.

Stebalien · 2019-05-06T22:32:33Z

Given libp2p/specs#80, we should probably just abort this whole thing and implement our own library that:

Implements that spec.
Doesn't have random race conditions.
Can handle things like network interface changes.

rargulati commented Feb 27, 2018

View reviewed changes

Kubuxu reviewed Feb 27, 2018

View reviewed changes

Stebalien mentioned this pull request Mar 3, 2018

Example with mdns discovery #282

Closed

Stebalien mentioned this pull request Jun 9, 2018

gx update go-log, sys, go-crypto ipfs/kubo#5100

Merged

2 tasks

Stebalien mentioned this pull request Jul 23, 2018

gx update deps ipfs/kubo#5268

Merged

rargulati and others added 3 commits July 23, 2018 12:00

Address PR feedback.

c5a5cf3

1. If we receive multiple ipv4 addresses, ensure that we convert them all to multiaddrs and store them in the peerstore. 2. Ensure we don't allocate a new resolver on every tick. Instead reuse this by storing it in our mdns type. 3. Attempt choosing a port.

fix gx imports

f498105

Stebalien force-pushed the swap-mdns-lib branch from c92f1f6 to f498105 Compare July 23, 2018 19:04

ghost assigned Stebalien Jul 23, 2018

ghost added the status/in-progress In progress label Jul 23, 2018

try to pick a decent port in mdns

528f157

Ideally, we'd dynamically announce ports based on the current interfaces. However, we'd like to merge this ASAP.

Stebalien added need/review Needs a review and removed status/in-progress In progress labels Jul 23, 2018

Stebalien added the status/blocked Unable to be worked further until needs are met label Jul 23, 2018

use fixed zeroconf gxified package

da471d9

ghost added the status/in-progress In progress label Jul 23, 2018

Stebalien removed the status/blocked Unable to be worked further until needs are met label Jul 24, 2018

Stebalien removed the status/in-progress In progress label Jul 24, 2018

Stebalien reviewed Jul 24, 2018

View reviewed changes

Stebalien requested a review from bigs July 24, 2018 00:14

bigs reviewed Jul 24, 2018

View reviewed changes

Stebalien added 3 commits July 24, 2018 21:20

don't concat txt records when handling dns-sd announcements

6651bb9

mdns: put all multiaddrs into a single peer info

aaa7f87

mdns: also handle IPv6 addresses

a7e8c11

ghost added the status/in-progress In progress label Jul 25, 2018

Stebalien added status/blocked Unable to be worked further until needs are met and removed status/in-progress In progress need/review Needs a review labels Jul 25, 2018

gpestana mentioned this pull request Jul 29, 2018

Be smarter when using MDNS #376

Open

Stebalien closed this May 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

discovery: swap mdns lib for grandcat/zeroconf #285

discovery: swap mdns lib for grandcat/zeroconf #285

rargulati commented Feb 27, 2018 •

edited by Stebalien

rargulati Feb 27, 2018

Kubuxu Feb 27, 2018

rargulati Feb 27, 2018

Kubuxu Feb 27, 2018

Kubuxu Feb 27, 2018

rargulati commented Feb 27, 2018

Kubuxu commented Feb 27, 2018

Stebalien commented Mar 1, 2018

rargulati commented Mar 5, 2018 •

edited

Stebalien commented Mar 5, 2018

Stebalien commented Mar 6, 2018

deckbsd commented Mar 24, 2018

Stebalien commented Mar 24, 2018 •

edited

Stebalien commented Jul 23, 2018

Stebalien commented Jul 23, 2018 •

edited

Stebalien Jul 24, 2018

bigs left a comment

bigs Jul 24, 2018

bigs Jul 24, 2018

Stebalien Jul 24, 2018

Stebalien commented Jul 25, 2018

Stebalien commented May 6, 2019

discovery: swap mdns lib for grandcat/zeroconf #285

discovery: swap mdns lib for grandcat/zeroconf #285

Conversation

rargulati commented Feb 27, 2018 • edited by Stebalien

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rargulati commented Feb 27, 2018

Kubuxu commented Feb 27, 2018

Stebalien commented Mar 1, 2018

rargulati commented Mar 5, 2018 • edited

Stebalien commented Mar 5, 2018

Stebalien commented Mar 6, 2018

deckbsd commented Mar 24, 2018

Stebalien commented Mar 24, 2018 • edited

Stebalien commented Jul 23, 2018

Stebalien commented Jul 23, 2018 • edited

Choose a reason for hiding this comment

bigs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Stebalien commented Jul 25, 2018

Stebalien commented May 6, 2019

rargulati commented Feb 27, 2018 •

edited by Stebalien

rargulati commented Mar 5, 2018 •

edited

Stebalien commented Mar 24, 2018 •

edited

Stebalien commented Jul 23, 2018 •

edited