Finished an initial version of GossipSub

[Mikerah]

I completed an initial version of gossipsub (I think). The code is based on this specification and the reference implementation in Go. The code most likely can't run due to several bugs that I have eyeballed and some missing functionality in other libp2p modules that I plan on working on.

I have started writing tests and I will be submitting PRs for these tests as I go along. In the meantime, any extra cleanup and bug fixes will be greatly appreciated.

Current blockers:

• Publish doesn't work. In the 2 node tests, the node that receives the message from a topic that it is subscribed to does receive a graft message but it doesn't send a I want message. As a result, it doesn't get the actual message.

[raulk]

@Mikerah – would you like @vasco-santos @jacobheun to have a peek as well?

[Mikerah]

@raulk If they have the time, that would be great!

[vasco-santos]

Great!
@Mikerah I will have a look early next week! 😀

[wemeetagain]

@vasco-santos
Added all of the above tests minus the floodsub interop test.
In working through that test, I believe that floodsub interop functionality will require some sort of (potentially minor) change to pubsub.
Specifically, even if we add this.libp2p.handle(constants.FloodSubID, this._onConnection) to the appropriate section of the gossipsub start method, the connection to a floodsub peer will error here because of the way pubsub only dials a single protocol here.

[Mikerah]

@wemeetagain For this test, what needs to be done is to make sure that floodsub peers added to a gossipsub peer's mesh receives a subscription as if it were a floodsub peer, not a gossipsub peer. So, from the floodsub peer's perspective, it cannot treat a gossipsub peer as a floodsub peer so a gossipsub peer has to treat the floodsub peer as it is.

So, instead of handling floodsub peers in the start method, what you can do is once a peer is connected, determine what protocol is it using. It's peerInfo object should have this.protocols populated with protocols that it supports. If that peer is a floodsub peer, just send a message to it as per the floodsub spec.

[wemeetagain]

@Mikerah
The problem is that a stream is never properly created. See the links in my previous comment.

I'm not talking about handling, in a general sense, the floodsub peers in the start method. I'm referring to the libp2p.handle method that gets called in the pubsub start method, which sets up a connection callback when a peer of a certain protocol is dialed. See here

The logic to maintain subscriptions and send messages appropriately to floodsub peers is already present (though untested).
The "issue" is that the pubsub module assumes the router has only a single protocol, and this is causing floodsub peers to not be properly connected, and this can only be fixed by refactoring pubsub or doing an unreasonable amount of method specialization in gossipsub.

[vasco-santos]

@wemeetagain that is indeed a problem! Can you PR js-libp2p-pubsub changing the libp2p.dialProtocol?

[wemeetagain]

@vasco-santos
How would you feel about merging in our current work and finishing out floodsub/golang/etc interop in other PRs?

stale review

[vasco-santos]

Thanks for the updates @wemeetagain

Can you check my latest comments? I will try to review the tests tomorrow, so that we can merge this PR and add the remaining tests in a new one

[vasco-santos]

Can you please complete the README.md first?

References:

• libp2p/js-libp2p-floodsub - it is not that cool documentation
• libp2p/js-libp2p-daemon-client - this is a more recent module with the docs as we intend to do from now on

[wemeetagain]

it looks like the API documentation was autogenerated in both cases.
I left the API section blank for now.

[vasco-santos]

LGTM! 🚀 It is time to merge this PR and iterate on it!

Awesome work @Mikerah and @wemeetagain 🎉

When you merge this PR, I will create a few issues in the repo with the remaining things we discussed here.

stale review