protocols/kad: Add double hasing for privacy to libp2p-kad

[umgefahren]

Description

To improve privacy during kademlia lookups it was proposed to add double hashing support to kad. The following part of the notion describes the changes pretty well:

Now

1. Client sends Req(CID) to DHT server nodes
2. DHT server nodes lookup if one of their provider records matches CID and if yes, return it
3. If not
   a. DHT server nodes compute Hash(CID)
   b. DHT server nodes look for the closest peerIDs (according to the XOR distance) to Hash(CID) in their routing table and return them to the client

After

1. Client sends (Req(Hash("CR_DOUBLEHASH" || MH)) to DHT server nodes, where MH is the MultiHash included in the CID
2. DHT server nodes lookup if one of their provider records matches Hash("CR_DOUBLEHASH" || MH) and if yes return it
3. If not
   a. DHT server nodes look for the closest peerIDs (according to the XOR distance) to Hash("CR_DOUBLEHASH" || MH) (without having to compute the hash of the CID themselves) in their routing table and return them to the client

Due to this change in behavior, the change necessary will be minimal but breaking. We have to figure out how we want to transition here. Also look at the corresponding notion document: https://pl-strflt.notion.site/Double-Hashing-Migration-Notes-75d723deb0d44a19a95615c5ccda3165

It probably makes sense to introduce these changes with prefix lookup and record encryption (also discussed in Notion Page by PL).

Relevant documentation:

• Notion Page by PL
• Go implementation: double hashing implementation ChainSafe/go-libp2p-kad-dht#1
  Related:
  • prefix lookup implementation ChainSafe/go-libp2p-kad-dht#2
  • provider record encryption ChainSafe/go-libp2p-kad-dht#3

Motivation

This change would increase reader privacy, by making it harder to track what a reader is looking for. It's part of the ongoing effort to increase privacy in libp2p.

Open Questions

• How do we want to transition?

Are you planning to do it yourself in a pull request?

Yes.

In the interest of making it easier, I would like to see #3130 to be resolved first.

[umgefahren]

Grant Proposal: filecoin-project/devgrants#1421

[guillaumemichel]

FYI: The reference Spec has moved from Notion to ipfs/specs#373