Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement peer discovery #69

Merged
merged 11 commits into from
May 24, 2019
Merged

Implement peer discovery #69

merged 11 commits into from
May 24, 2019

Conversation

albrow
Copy link
Contributor

@albrow albrow commented May 23, 2019

This PR adds support for peer discovery via go-libp2p-kad-dht and go-libp2p-discovery. It also adds a new environment variable USE_BOOTSTRAP_LIST. If set to "true", it uses the IPFS default bootstrap list to bootstrap the DHT.

WIP right now because of some weird behavior when using the bootstrap list. If you set it to "false", you can test peer discovery by following these steps:

  1. Create three 0x Mesh nodes (A, B, and C).
  2. Manually connect A to B via the AddPeer RPC method.
  3. Manually connect B to C via the AddPeer RPC method.
  4. Observe that A is introduced to C through B and they will connect to each other.

@@ -166,7 +166,7 @@ func (w *Watcher) pollNextBlock() error {
if err == ethereum.NotFound {
log.WithFields(log.Fields{
"blockNumber": nextBlockNumber,
}).Info("block header not found")
}).Trace("block header not found")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fabioberger This log was getting pretty noisy. I think it is always expected to happen while we are waiting for the next block to be mined. Are you okay with changing this to Trace?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah totally, thanks for making that change. It is an expected condition.

@@ -253,6 +388,5 @@ func (n *Node) receive(ctx context.Context) (*Message, error) {
// Close closes the Node and any active connections.
func (n *Node) Close() error {
n.cancel()
n.sub.Cancel()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're passing in n.ctx to n.sub during initialization so it now closes automatically when we call n.cancel.

Copy link

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some feedback off the top of my head. Let me know if you'd like a deeper review!

core/node.go Outdated Show resolved Hide resolved
core/node.go Show resolved Hide resolved
wg.Add(1)
go func() {
defer wg.Done()
if err := n.host.Connect(connectCtx, *peerInfo); err != nil {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connect will establish a connection and trigger the notifees on protocols like the DHT. Those protocols will in turn open streams to the connected peer to test whether the other party supports the protocol, and will perform any init logic in consequence (e.g. adding the peer to a routing table). Until that happens, you won't have a working DHT node. Since the notifees are triggered asynchronously, so you should add like a 2 second wait after all dials completed.

We are speccing out a local event bus (will tag you in that issue) that will allow us to be deterministic instead of probabilistic in the near future (you'll listen to an event instead of relying on a timing condition), but until that's a thing, this is probably the safest path.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I'll add a 2 second wait after dialing all the bootstrap peers for now. Also wanted to point out that the chat-with-rendezvous example doesn't do this: https://github.com/libp2p/go-libp2p-examples/blob/master/chat-with-rendezvous/chat.go. Should we add a time.Sleep there as well?

core/node.go Show resolved Hide resolved
core/node.go Outdated Show resolved Hide resolved
core/notifee.go Show resolved Hide resolved
Instead of protecting local connections, we now tag peers who speak
the 0x-mesh-floodsub protocol. This gives them a slightly positive score
and makes them less likely to be removed by the Connection Manager.
@albrow albrow changed the title WIP: Implement peer discovery Implement peer discovery May 23, 2019
@albrow
Copy link
Contributor Author

albrow commented May 23, 2019

@fabioberger removed WIP tag. We will probably make improvements to peer discovery and peer ranking over time, but this PR is ready for final review.

@albrow albrow requested a review from fabioberger May 23, 2019 22:25
@albrow
Copy link
Contributor Author

albrow commented May 23, 2019

Fixes #72

core/notifee.go Outdated
func (n *notifee) OpenedStream(network p2pnet.Network, stream p2pnet.Stream) {
go func() {
// HACK(albrow): When the stream is initially opened, the protocol is not set.
// For now, we have to manually poll until it is set.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a PR or issue you could add a link to?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added a link to this issue: libp2p/go-libp2p#467. I'm not 100% sure if it will help us avoid this hack; it depends on how exactly the event bus is implemented.

@albrow albrow merged commit 7b7c4e1 into master May 24, 2019
@albrow albrow deleted the feature/peer-discovery branch May 24, 2019 00:12
@albrow albrow mentioned this pull request May 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants