Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: do not use identify protocol, and actually use the nodes from the config for peer exchange #5212

Merged
merged 1 commit into from
May 27, 2024

Conversation

richard-ramos
Copy link
Member

  • Identify protocol usage is no longer needed, as invoking the Connect method from go-libp2p triggers the identify protocol anyway
  • The automatic disconnection that happened before as part of identifyAndConnect is removed. The connection manager of go-libp2p will automatically disconnect peers once we reach a high watermark. This should help in the case of light clients, since we will no longer connect to a peer, disconnect from it, and attempt to connect to it again.
  • I noticed that the comment of the peer exchange loop did not match the actual behavior of the code. I modified it so the peer exchange request are done against the nodes in the configuration

@status-im-auto
Copy link
Member

status-im-auto commented May 22, 2024

Jenkins Builds

Click to see older builds (20)
Commit #️⃣ Finished (UTC) Duration Platform Result
✖️ 1870f00 #1 2024-05-22 19:22:20 ~1 min tests 📄log
✔️ 1870f00 #1 2024-05-22 19:25:33 ~4 min linux 📦zip
✔️ 1870f00 #1 2024-05-22 19:26:13 ~5 min ios 📦zip
✔️ 1870f00 #1 2024-05-22 19:26:45 ~5 min android 📦aar
✔️ 7da01c4 #2 2024-05-22 19:41:46 ~1 min android 📦aar
✔️ 7da01c4 #2 2024-05-22 19:42:28 ~2 min linux 📦zip
✔️ 7da01c4 #2 2024-05-22 19:43:11 ~3 min ios 📦zip
✔️ 7da01c4 #2 2024-05-22 20:23:53 ~43 min tests 📄log
✖️ e81702e #3 2024-05-23 19:24:59 ~1 min tests 📄log
✔️ e81702e #3 2024-05-23 19:26:26 ~2 min linux 📦zip
✔️ e81702e #3 2024-05-23 19:26:34 ~2 min android 📦aar
✔️ e81702e #3 2024-05-23 19:27:34 ~3 min ios 📦zip
✖️ 7cea1b6 #4 2024-05-24 14:34:17 ~1 min tests 📄log
✔️ 7cea1b6 #4 2024-05-24 14:36:34 ~3 min ios 📦zip
✔️ 7cea1b6 #4 2024-05-24 14:37:11 ~4 min linux 📦zip
✔️ 7cea1b6 #4 2024-05-24 14:38:47 ~5 min android 📦aar
✖️ 0328d23 #5 2024-05-27 00:12:27 ~1 min tests 📄log
✔️ 0328d23 #5 2024-05-27 00:13:53 ~2 min linux 📦zip
✔️ 0328d23 #5 2024-05-27 00:14:07 ~2 min android 📦aar
✔️ 0328d23 #5 2024-05-27 00:16:31 ~5 min ios 📦zip
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 31104b1 #6 2024-05-27 12:59:13 ~1 min android 📦aar
✔️ 31104b1 #6 2024-05-27 13:00:08 ~2 min linux 📦zip
✔️ 31104b1 #6 2024-05-27 13:02:47 ~5 min ios 📦zip
✔️ 31104b1 #6 2024-05-27 13:42:25 ~44 min tests 📄log

Copy link
Contributor

@chaitanyaprem chaitanyaprem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left few comments.

But like that we are removing identify logic at this layer.

wakuv2/waku.go Outdated

// Attempt to connect to the peers.
// Peers will be added to the libp2p peer store thanks to identify
go w.connect(w.ctx, discoveredNode.PeerInfo)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, This looks like the changes i was doing while integrating new Filter-mgmt into status-go as test keeps failing.
Thanks for this.

wakuv2/waku.go Outdated Show resolved Hide resolved
wakuv2/waku.go Outdated Show resolved Hide resolved

// Attempt to connect to the peers.
// Peers will be added to the libp2p peer store thanks to identify
go w.connect(discoveredNode.PeerInfo, wps.DNSDiscovery)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be added to peer store already with discv5 right? wondering why make such connection in peer exchange loop.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason why I'm trying to connect again is because DNS Discovery might have failed for one of the enrtrees, so what this code does is attempt to connect to all nodes, but assuming the nodes are already connected, it should be fine, since no new connection will be made.
But I will improve this conde by checking if the peer is already connected to skip this step.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't matter, because invoking connect on already connected peer doesn't do anything.
It only does a dial if there is no connection to that peer. Reference libp2p code.

https://pkg.go.dev/github.com/libp2p/go-libp2p@v0.32.2/core/host#Host.Connect

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be added to peer store already with discv5 right? wondering why make such connection in peer exchange loop.

In case of lightClient discv5 won't be there, so these peers that are discovered using dnsDiscovery should be added to and connected explicitly.

wakuv2/waku.go Outdated
// The peer connector in go-waku uses Connect, so it will execute identify as part of its
connectedness := w.node.Host().Network().Connectedness(peerInfo.ID)
if connectedness == network.NotConnected || connectedness == network.CanConnect {
w.node.AddDiscoveredPeer(peerInfo.ID, peerInfo.Addrs, origin, nil, true)
Copy link
Contributor

@chaitanyaprem chaitanyaprem May 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if pubsubTopics list should not be empty rather we should use defaultShardedPubsubTopic here.
Because if the node is lightClient, then pubsubTopic list won't be populated for the peers in peerstore and will lead to an issue in peer selection.
WDYT @richard-ramos

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I'll introduce that change. Nodes for the network should still support the default shard pubsub topic so it makes sense to use it here

}
}
func (w *Waku) connect(peerInfo peer.AddrInfo, origin wps.Origin) {
// Connection will be prunned eventually by the connection manager if needed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One doubt i have is, we have keepAlive check going for all peers. If so, would there by any pruning happening by connection manager?
And peers that get connected here, would never be disconnected right.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I do not know tbh. I'll look into what criteria is used by the connection manager to prune connections. I do remember reading that once it reaches the high watermark it will prune nodes until it reaches the low watermark. I wonder if doing a ping affects the criteria somehow?

Copy link
Contributor

@chaitanyaprem chaitanyaprem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Apart from the one comment that needs to be addressed, but not part of this PR. Maybe a separate PR.

@chaitanyaprem
Copy link
Contributor

@richard-ramos looks like conventional commit check is failing which is causing tests to not get executed.
Maybe we can add that check separately to github checks so that tests can proceed even if there are commit naming issues.

Looks like _ has to be used to indicate non-breaking changes. e.g: fix_ : <message>

@richard-ramos
Copy link
Member Author

Looks like _ has to be used to indicate non-breaking changes. e.g: fix_ :

uff, i always forget this. Sadly, in status-go, there's nothing conventional about the format used for conventional commits ;)

@richard-ramos richard-ramos merged commit 0a9cff2 into develop May 27, 2024
12 checks passed
@richard-ramos richard-ramos deleted the identify-peer-exchange branch May 27, 2024 13:43
chaitanyaprem pushed a commit that referenced this pull request May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants