using mDNS and DHT at the same time

[daviddias]

I'm trying to figure out what is the protocol to achieve mDNS and DHT interop so that exchange doesn't have to understand the way that the resource/peer is found. Mainly my questions are:

• Do we populate the DHT routing table with mDNS entries found?
  • If yes, what is the criteria to drop/keep these mDNS entries? Or do we overload the protocol to have a virtual infinite bucket for mDNS entries? What if this value is abused?
  • If not, what is the criteria to opt for mDNS? Do XOR metric to all of the peers (mDNS available + DHT) and simply send to whatever is close? Is there any criteria in this process to prefer local network peers over remote ones?

[jbenet]

i see the routing systems as fundamentally different, you would resolve over one AND/OR the other. so an ipfs distribution that uses DHT content routing and mdns content routing might see things in one and not the other, and could opt to advertise certain content in one and not the other.

there can also be policies for "bridging" routing systems in a way, but this has to be very well thought out (it's tricky to get right as you point out, and in light of the below).

some important reasons for separating routing systems is use cases like:

1. high performance routing inside private (or semi-private, just different) network
2. private routing of one's own content between one's own nodes (e.g. i dont want to advertise my private files to the world, but only to a set of nodes i control/trust)
3. private "web-of-trust" routing strictly with trusted peers. (e.g. i want to advertise + route certain content over groups i trust) (functionally same as 2., but extends to a PKI chain model)
4. privacy-focused messaging over some from of Oblivious Routing

[jbenet]

BTW, using these for content routing is different from using them just for peer discovery. today we use mdns only for peer discovery, but plan to extend to do content routing too.

(i need to write so much spec. ooof.)

[daviddias]

I finally understood (thought I had before though) what you exactly meant with "mdns only for peer discovery" after your clarification on this issue: ipfs/kubo#1396 (comment) woot! :) Going to leave it written down here as sorts of documentation for later.

I had in my mind that mDNS could be used to find better candidates to "forward/ping" query messages, but now I got it. In essence, we have 3 mechanisms for discovery"

• mDNS
• bootstrap peer list (distributed with ipfs impl itself)
• random walking on the network https://github.com/ipfs/go-ipfs/blob/master/routing/dht/dht_bootstrap.go

Which means that even if the "destPeer" for our query is in our LAN, if it didn't fulfil the requirement to be in our kbucket (not "close enough by xor metric"), we will talk with it after some DHT based queries. Although it would be cool for it to figure out that it is right on the side of the peer it needs to talk, I understand that from the implementation stand point, this save a lot of conditions and edge cases.

Thanks!