Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement peer-discovery interface #222

Open
ghost opened this issue Aug 29, 2017 · 18 comments
Open

Implement peer-discovery interface #222

ghost opened this issue Aug 29, 2017 · 18 comments

Comments

@ghost
Copy link

@ghost ghost commented Aug 29, 2017

As per https://github.com/libp2p/interface-peer-discovery

This interface would be implemented by:

  • mdns
  • bootstrap
  • random walk (does this currently exist in go-libp2p?)
  • in the future: webrtc, ...
@vyzo
Copy link
Contributor

@vyzo vyzo commented Aug 29, 2017

do we want this to be dht based?
It could be the solution to the generalized rendezvous problem, and be usable for pubsub, relay, etc

@Kubuxu
Copy link
Member

@Kubuxu Kubuxu commented Aug 29, 2017

Additional provider of peer-discovery is "old peers". This way we could make the network independent from the bootstrap nodes after first run.

@daviddias
Copy link
Member

@daviddias daviddias commented Aug 29, 2017

do we want this to be dht based?

DHT is Peer Routing + Content Routing. Read:

Peer Discovery is things like MulticastDNS or Bootstrap. Read:

peer-discovery is "old peers"

Yeah, we can have a resurrect Peer Discovery that knows how to store to disk old peers and then brings them back on node boot. 👍 👍

@deckbsd
Copy link

@deckbsd deckbsd commented Feb 22, 2018

Hello everybody,

If i understand correctly, does that mean that actually from a go-libp2p peer i can't try to find some other peers ? that's why i was wondering how, when i start a host i can't make that host to connect to any other nodes. Because actually in the examples, each nodes are created like in client/server basic model (the client know the ip of the "server"(an other node that it can communicate with). m i wrong ?

@Kubuxu
Copy link
Member

@Kubuxu Kubuxu commented Feb 23, 2018

If i understand correctly, does that mean that actually from a go-libp2p peer i can't try to find some other peers ?

No, finding peers works (it is sometimes difficult in case of complicated/layered NATs).

This issue is about internal go-libp2p interface allowing better code structure when it comes to peer discovery.

@agahEbrahimi
Copy link

@agahEbrahimi agahEbrahimi commented Apr 15, 2018

@Kubuxu Are there any examples of the peer discovery? I've been wandering around for 2-3 days trying to find an example and only found one in JS https://github.com/libp2p/js-libp2p/tree/master/examples/discovery-mechanisms which was also mentioned above, but none in go.
Even though the website says that they are all complete in go.
screen shot 1397-01-26 at 10 47 01

Thank you

@Kubuxu
Copy link
Member

@Kubuxu Kubuxu commented Apr 17, 2018

@agahEbrahimi most of them should work by default. We are working on clearing up the bootstraping but for an example you can look here: #302

Random walk (DHT) should work by default and so should mdns AFAIK.

@daviddias
Copy link
Member

@daviddias daviddias commented Nov 5, 2019

Can someone point me to an up to date example of how to enable/disable random-walk in go-libp2p?

@daviddias
Copy link
Member

@daviddias daviddias commented Nov 5, 2019

Just did a quick "chase the PR" exercised and discovered that:

Still looking for random-walk

@daviddias
Copy link
Member

@daviddias daviddias commented Nov 5, 2019

@aschmahmann I would say no but given that there is a bunch of terms that are overloaded, the best is for me to clarify my intent: What I want to achieve is to enable/disable the routine that searches the DHT for a random peer (most likely that doesn't exist) with the purpose of refreshing the routing tables. Is there a way to enable/disable this (known in the P2P literature as random-walk)?

@aschmahmann
Copy link
Contributor

@aschmahmann aschmahmann commented Nov 5, 2019

Why do you think that it's not enough? The comments at https://github.com/libp2p/go-libp2p-kad-dht/blob/4fd64988b55d6e93405eceffeb3037da46d1848b/dht_bootstrap.go#L48 seem to imply that it's exactly what you're asking for.

@yusefnapora
Copy link
Contributor

@yusefnapora yusefnapora commented Nov 5, 2019

@daviddias I think from reading through the kad bootstrap code a bit, that "bootstrap" in the context of the libp2p kad DHT does mean random-walk.

It looks like by default it does a query for a random peer id in each bucket that hasn't been refreshed within some interval, plus a periodic query for our own id.

If you set the DisableAutoBootstrap option that @aschmahmann mentioned, it looks like it won't do the bootstrapping on a periodic timer, but it would still be possible to manually trigger the bootstrap process by calling the Bootstrap method.

@daviddias
Copy link
Member

@daviddias daviddias commented Nov 5, 2019

@daviddias I think from reading through the kad bootstrap code a bit, that "bootstrap" in the context of the libp2p kad DHT does mean random-walk.

Aaaaaaah!!! Ok, so in go-libp2p bootstrap gets used for:

  • Connecting to set of bootstrap nodes when the node boots up (aka railing into the network in p2p literature)
  • Refreshing the Kademlia Routing Tree every now and then (aka random-walk in the p2p literature)

It looks like by default it does a query for a random peer id in each bucket that hasn't been refreshed within some interval, plus a periodic query for our own id.

How does it actually achieve this? Given that we sha2-256 the peerIds for the Kad Routing Table in our DHT implementation, one would actually have to brute force to find another PeerId that would fall within a specific k-bucket.

Anyhow, given that random-walking is walking randomly over new ways, most k-buckets will get somewhat refreshed along the way.

If you set the DisableAutoBootstrap option that @aschmahmann mentioned, it looks like it won't do the bootstrapping on a periodic timer, but it would still be possible to manually trigger the bootstrap process by calling the Bootstrap method.

Thank you! This is exactly what I needed. ❤️ @aschmahmann @yusefnapora

@aschmahmann
Copy link
Contributor

@aschmahmann aschmahmann commented Nov 5, 2019

How does it actually achieve this? Given that we sha2-256 the peerIds for the Kad Routing Table in our DHT implementation, one would actually have to brute force to find another PeerId that would fall within a specific k-bucket.

Sooo.... funny story about that we basically pre-brute force 1 per bucket for the first 16 buckets.

Precomputed buckets: https://github.com/libp2p/go-libp2p-kbucket/blob/8b77351e0f784a5f71749d23000897c8aee71a76/bucket_prefixmap.go#L4

Reasoning: https://github.com/libp2p/go-libp2p-kbucket/pull/38/files#r309983750

@daviddias
Copy link
Member

@daviddias daviddias commented Nov 5, 2019

🤯

Clever though!

@daviddias
Copy link
Member

@daviddias daviddias commented Nov 5, 2019

@aschmahmann @yusefnapora your tips worked like a charm testground/testground@c874df0 . Thank you! ❤️

@aarshkshah1992
Copy link
Collaborator

@aarshkshah1992 aarshkshah1992 commented Nov 6, 2019

For posterity:

@aschmahmann's suggestion is bang on. Please refer to this writeup for a good explanation on how to use the "Dht random walk"/"Dht refresh" feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants