Mapping for alternate hashes (hash index)

[btrask]

(Moved from ipfs/kubo#2098)

On Tuesday @jbenet was so kind as to spend a couple hours talking to me, where I got a chance to voice some of my concerns regarding ipfs/kubo#1953.

It seemed like one aspect Juan was amenable to was a way of looking up blocks (and possibly files) by hashes besides the primary hash (SHA-2 256) used by IPFS internally.

Features Juan wanted to see:

• An index implemented as part of the DHT for going from hash -> hash
• Flexibility for adding new hash algorithms (and dropping old ones?) over time (e.g. moving to SHA-3)
• Nodes should not be required to compute/verify all hashes themselves, for efficiency
• Some sort of trust/reputation system to weed out bad hashes and bad actors
• Ultimately, the index will not be perfectly reliable and end recipients will probably want/need to verify the data for themselves

Features I wanted to see:

• Hashes of file content (not just block content) mapping to root blocks (Hash conversion for import/export and long term archival kubo#1953)
• A first-class interface and namespace (my suggestion was /ipls/[hash], for "link system")
• Emphasis on this mode for file-centric (rather than block-centric) uses (e.g. ipfs add returning a file hash rather than a root block hash)

We agreed that this system should not be built on top of IPNS. Hashes shouldn't rely on owners who have control of them, and don't need to be mutable.

It sounded like this was a fairly distant goal, but I figured we should at least start documenting our ideas.

Thanks again Juan for talking!

[harlantwood]

Excellent. Much needed in the long term. Thanks for documenting.

[davidar]

👍

[ghost]

This might also be useful for supporting other tools' public-key schemes

[btrask]

Yes, the same index system could work for both, but I hope the address spaces are kept separate (similar to /ipfs/ versus /ipns/).

[keks]

I'm proposing to use something along the lines of https://github.com/dedis/cosi for this, together with a lightweight reputation system.