IPLD Resolver: Membership

[aakoshh]

Part of #475

We want to know which IPC agents we can contact to resolve content from a specific subnet. To do so, we can use Gossipsub.

First, I'm assuming we have a single P2P network of agents, with everyone potentially able to connect to anyone else, with every agent running a single Swarm on a single address, serving the needs of all subnets. Later we can consider alternatives.

There are at least two ways to use Gossipsub to achieve what we want:

• Agents who are able to serve data from a subnet can publish a message into a topic, e.g. /ipc/subnet-agents. The message will spread across the network, and every node can build up a view of which agents do which subnets (a bipartite graph, as agents can participate in multiple subnets).
• We can potentially forego even publishing if we use the topics themselves to signal membership. Agents who are able to serve data from a subnet can subscribe to a topic, e.g. /ipc/subnet/<subnet-id>. The subscription information spreads around the network and at least the Rust version of Gossipsub can be queried to list peers subscribed to any topic. The parent subnet agents would not subscribe to the child subnets, they would just query their Gossipsub behaviour who are the subscribers of a child subnet, and contact them when they need to. The drawback here is that we only learn the subscriptions of peers we are connected to, wheras with the first approach we learn about everyone.

Through Gossipsub, we would learn about PeerIds that are serving subnets, but not their address. To connect to them, we rely on #34 to learn their address.

In this PR, we develop a Membership behaviour, such that:

• It wraps a Gossipsub instance
• It has a join and a leave method that the agent can call to subscribe or publish to the necessary topics to signal membership to others
• It listens to events from Gossipsub and either maintains a list of peer-subnet mappings, or is able to ask Gossipsub for the information (if it has to maintain it, we need to think about limits on the size of the collection).
• (Optional) It raises events when it learns about new members in a topic, which could be used to instruct the Discovery behaviour to ensure we proactively learn their addresses, unless they are already known, by doing Kademlia lookups.
• It provides a members method to return the list of peer IDs of data providers in a subnet.
• If we publish, then periodically re-publish the current subnet membership information about the agent itself, to make sure new joiners are informed. (We could publish a list, which would mean we don't even need join/leave, just a serve message with all the subnets, and a timestamp).
• If we publish, then track the last time we heard about the membership of an agent, and stop suggesting connecting to it if it's been too long - this is like a heartbeat so we don't end up trying to connect to agents long gone.

Questions:

• How does a new joiner learn about the current memberships? Does Gossipsub receive subscriptions from peers it connects to? Should we periodically publish our memberships to keep them fresh in everyone's tables?
• How does Gossipsub decide who to connect to in the first place? There is an option for explicit peers (we could use the same list as we do for bootstrapping Kademlia, but it looks like it also tries to connect to anyone whenever a connection is established by the Swarm, so the natural workings of Kademlia queries will prompt Gossipsub to connect as well.

Overall, the explicit publishing of messages is probably better, because:

• The Go implementation would not serve subscription lists, it would only work for Rust.
• It allows us to learn about peers we aren't connected to at the moment, but we might want to if we need their data, as opposed to only knowing about, say, the 50 agents we are actually connected to.