IPNS over PubSub as an independent transport

[aschmahmann]

IPNS over PubSub is not an independent transport from the standard IPNS over DHT. In particular, the first time a user tries to do an IPNS resolution when IPNS over PubSub is enabled it hits the DHT, and only on subsequent resolutions is PubSub utilized.

To accomplish this we need:

• A mechanism to efficiently support persistence for PubSub
  • Use a persistent pubsub router by making gossipsub more configurable (Added subscription specific configurations to PubSub libp2p/go-libp2p-pubsub#171). Replaced with alternative proposals below.
  • Use validators + rebroadcasting in go-libp2p-pubsub-router to enable efficient persistence (Add Persistence Layer on top of PubSub libp2p/go-libp2p-pubsub-router#33)
  • Have Peer Join events coming from PubSub (Add the ability to handle newly subscribed peers libp2p/go-libp2p-pubsub#190)
  • Utilize the Peer Join events to ask for the latest state from peers so that we don't need to frequently rebroadcast (Add Persistence Layer on top of PubSub libp2p/go-libp2p-pubsub-router#33)
• An easy to use mechanism for discovering peers in PubSub that is not tied to the DHT/content routing
  • e.g. As of now IPNS uses go-libp2p-pubsub-router which uses content routing to bootstrap the DHT
  • Utilize discovery interfaces in PubSub (Add Discovery libp2p/go-libp2p-pubsub#184)
  • Have multiple discovery interfaces available for use (to prove out the interface)
    • Rendezvous service (Implement rendezvous protocol spec libp2p/go-libp2p-rendezvous#1)
    • Rendezvous discovery client (Rendezvous compatibility with Discovery interface libp2p/go-libp2p-rendezvous#2)
• Integrate improved IPNS over PubSub into go-ipfs
  • Change the applicable IPNS value routers to support querying both DHT and Rendezvous
  • Add Rendezvous support into go-ipfs
  • Create a caching discovery mechanism that can use both DHT and Rendezvous servers and utilize that in pubsub-value-router (Add Backoff Cache Discovery libp2p/go-libp2p-discovery#26)
  • Bubble up all PubSub changes into go-ipfs (IPNS over PubSub as an Independent Transport #6758)

Some benefits of IPNS over PubSub as an independent transport from IPNS over DHT.

1. DHT performance issues are either less problematic or not relevant for us (depending on the discovery mechanism used)
2. We mostly get IPNS republishing for free (i.e. even if the publisher goes offline, other people can still make the IPNS record available). This means we can largely mitigate IPNS records that are abandoned by their authors since, as long as someone who cares about the record is maintaining and advertising it, the record will be available.
   • This is related to ipfs name follow #4435, since we would need a CLI method to indicate we want to pin an IPNS (public) key.

Demo and brief explanation of IPNS over PubSub as an independent transport (2m40s).

• https://www.youtube.com/watch?v=XniIDIXU8RE

^ @autonome @lidel @Stebalien @momack2

Update: Given that the rendezvous work has been put on hold by libp2p and that it's not actually necessary for PubSub to work as an independent transport those work items are now closed.

However, since we do already have MDNS discovery support (albeit not through the libp2p Discovery interfaces) IPNS over PubSub can function even in the total absence of a DHT.

[aschmahmann]

The only remaining issue is not with IPNS over PubSub as an independent transport, but with how we handle the multiple transports at an API level (ipfs/specs#198 (comment)).

This issue is closed for now and any Issue/PR dealing with the above will link to this issue.